Hermes Agent Nearly Doubled Speech-to-Text Accuracy on Hard Words: 99 to 183 of 240
- Level Advanced
- Setup1–3 hrs (estimated)
- Evidence Strong evidence
- CostNot stated in source
- No-codeNo
What they did
A user turned his 15K saved Wispr Flow dictations into a test set for the local Whisper model behind his Hermes Agent. The agent turned his Wispr dictionary into a hint list, diffed raw vs corrected text to find words Whisper keeps mishearing, made find-and-fix rules, and tested every rule against the full history. With no fine-tuning, dictionary words went from 99/240 to 183/240 on Wispr clips and 19/45 to 43/45 on sentences he read aloud. It now archives the data nightly and looks for new fixes weekly.
How it went
Dictionary words right: 99/240 to 183/240 (Wispr clips) and 19/45 to 43/45 (read-aloud sentences).
Worth knowing
- Detailed single-user write-up with before/after numbers and honest misses: giving Whisper the hint list directly sometimes dropped whole sentences, and one rule would have changed his daughter's name Quinn to Qwen. Wispr deletes audio after 14 days, so archive first.
Prompt
Cleaned version rewritten from the source — not a direct quote.
My dictation app saves what the speech model heard and what it was corrected to: [folder]. Use it to improve my local [Whisper] setup without fine-tuning: turn my custom dictionary into a hint list, diff raw vs corrected text to find words it keeps mishearing, and propose find-and-fix rules. Test every rule against the full history and reject any that break normal speech. Report before/after accuracy on a held-out test set, and archive the history nightly so it isn't lost.
Recommended
Try it in Grok Bot
Originally done with Hermes Agent; this prompt works in Grok Bot.
- Copy the prompt (button below).
- Open Grok Bot chat and paste it. Replace any [bracketed] placeholders.
- Connect the listed tools (as needed) as connectors when Grok Bot asks.
- If the task repeats, ask: "Make this a daily routine."
Alternatives: the same prompt works in Hermes Agent, Claude, ChatGPT Agent, OpenClaw.
Set it up
Stack: Hermes Agent + local Whisper large-v3-turbo + Wispr Flow history
- Read the original source for full context.
- Connect the tools your agent needs.
- Paste the prompt and replace [bracketed] placeholders.
- Run once manually and check the output before scheduling it.
Generic suggested steps, not from the source.
Evidence
Detailed single-user write-up with before/after numbers and honest misses: giving Whisper the hint list directly sometimes dropped whole sentences, and one rule would have changed his daughter's name Quinn to Qwen. Wispr deletes audio after 14 days, so archive first
Source: X · 2026-10-16 · Verified by us: not yet · Report broken or outdated