What we changed, and what this page is for
In October 2026 we retuned the recognition engine so it loses far fewer words, especially on long recordings — that is the change that matters most when you actually use the tool. This page still shows the plain word-error rate, measured honestly on the same retuned engine and including the clips that went badly. The word-error rate is not the headline here; the headline is that the transcript keeps more of your words. The full drop-and-hand-fix breakdown, including three long files, is on the transcription benchmark.
Accuracy by condition
All eight clips were run through both models this tool offers — Moonshine base (the English default, quantised) and whisper-small (the optional higher-accuracy tier, and the model used for other languages) — on 2026-10-03, and the output was compared word by word against a reference transcript. Every word that came back wrong, missing or added was counted as an error.
On the audio most people actually have — the four clean and light-noise real-speech clips, 85 reference words — the default English model is 7.1% wrong and the optional small tier is 8.2% wrong. The heavier buckets below are deliberately harsh edge cases, not your everyday recording.
- clean synthetic speech — 1 clip, 33 words — Moonshine 6.1% wrong (2 words); small 0.0% (0 words)
- real speech, no added noise — 2 clips, 28 words — Moonshine 7.1% wrong (2 words); small 7.1% (2 words)
- real speech with light background noise — 2 clips, 57 words — Moonshine 7.0% wrong (4 words); small 8.8% (5 words)
- real speech with heavy background noise — 2 clips, 89 words — Moonshine 21.3% wrong (19 words); small 16.9% (15 words)
- real speech, telephone band with echo and hiss — 1 clip, 22 words — Moonshine 18.2% wrong (4 words); small 0.0% (0 words)
- typical files only (clean + light noise real speech, 4 clips, 85 words) — Moonshine 7.1% wrong (6 words); small 8.2% (7 words)
- all eight clips (stress clips included) — 229 words — Moonshine 13.5% wrong (31 words); small 9.6% (22 words)
The 13.5% is not the average of the five rows. It is 31 error words divided by 229 reference words, so the longer buckets weigh more. Adding the per-bucket percentages and dividing by five gives a different and wrong number. The small tier's 9.6% is counted the same way (22 error words ÷ 229).
A higher-accuracy tier is built in. The same eight clips run through whisper-small (the optional tier) came back 9.6% wrong overall — against 13.5% for the English default — and it drops far fewer words: 1 word lost against 3 for the English default. The full bucket-by-bucket and tier-by-tier breakdown, including the three long files and a cloud reference, is on the transcription benchmark. Even there we say "about the same", not "more accurate": the cloud reference drops fewer words than small, so neither local tier wins outright.
The worst clip, on its own
One clip accounts for most of the damage. E1-real-noise-heavy is 29.4 seconds of real speech with heavy background noise, 71 reference words, and on the English default model it came back with 18 of those 71 words wrong — 25.4%. The other heavy-noise clip in the same bucket (E2, 18 words) came back at 5.6%. Same condition, same model, 20 points apart.
That spread is the honest part of this page. A single averaged accuracy figure hides it: if we had only run E2, we would have reported a heavy-noise number four times better than the one E1 actually produced.
The optional small tier does better on this clip: on E1 it came back 18.3% wrong (13 words) rather than 25.4% — still the worst clip, but it loses fewer words. Across the set the two models trade places: the English default is the more accurate on typical clean and light-noise speech, and whisper-small is the more accurate on the deliberately harsh clips and is the model used for other languages.
What this means if you are about to use the tool
- Clean speech is close to usable as-is. 6.1% on the clean clip with the English default, and the two clips with no added noise averaged 7.1%. The small tier got the clean clip exactly right.
- Heavy background noise is where it breaks. 21.3% across the two heavy clips with the English default, and 25.4% on the worst one. Expect to read the transcript rather than trust it. The small tier cuts that bucket to 16.9%, which is what the optional tier is for.
- Telephone-band audio with echo is middling. 18.2% on the one clip in that bucket with the English default — worse than clean, better than heavy noise. The small tier got that clip exactly right.
- Read the result before you use it. Numbers, names and technical terms are what you should check yourself; those are the words a speech model gets wrong most often.
What we did not test
- Eight clips, not eight hundred. Every percentage above rests on 1–2 clips per bucket. The per-bucket numbers are indicative, not precise.
- One speaker per clip, English. No multi-speaker recordings, no overlapping speech, no accents beyond the ones in these eight clips.
- One noise type. Both noisy buckets use pink noise. Traffic, music and room babble were not tested.
- Chinese was measured separately, on a single 23-second clip, and the model emits Traditional characters where the reference is Simplified — so a raw character comparison is meaningless there. That result is in the dataset, not on this page.
Where the raw data is
Every number on this page comes from two files: wer-by-condition.csv (the five buckets plus the total) and wer-by-sample.csv (all eight clips individually, with the per-clip error counts). Both are published, along with the measurement scripts and the reference transcripts, so the arithmetic above can be checked or re-run.
- Raw data and the measurement scripts — github.com/supersophia8888-cloud/clipquill-asr-benchmark
- Archived copy with a DOI — 10.5281/zenodo.22826968, the concept DOI: that link always resolves to the newest version of this dataset, all versions.
- The same data as a dataset — huggingface.co/datasets/sophia8888/clipquill-asr-benchmark
Run it on your own file
The tool is on the front page of this site. Drop in an audio or video file you already have, pick the spoken language, and read the transcript. Nothing is uploaded — the model runs in your browser tab and the file never leaves your device. English runs on Moonshine base and the other languages on whisper; a larger, more robust tier (whisper-small) can be switched to before running a file.
The same clips, counted as editing load
The eight clips above are also scored by how many words were dropped and how many spots you would fix by hand, alongside three longer files and a cloud reference — see the transcription benchmark.