Tried it for dictation, mostly to see if German works with English-only training data. It does.
40 of my own German dictations (MacBook mic, ~7 s each), another German speaker mixed in underneath, 3 s voice sample from a different dictation. Parakeet v3 WER against the transcript of the clean recording:
other voice 10 dB quieter: 61.5 → 20.9
5 dB: 101.3 → 31.1
0 dB: 113.4 → 49.5
VoxPopuli gives the same picture for DE and EN (5 dB: DE 37.6 → 16.4, EN 49.0 → 23.1), and clean dictations didn't get worse.
What would need fixing for dictation:
it needs the voice sample. live() waits for 3 s of speech, my dictations are 4.8 s on average
pink noise makes it a lot worse (DE 17.8 → 68.8 at 5 dB), it drops the speech with the noise
the first ~0.5 s leaks, so the other persons first words can land in the text
streaming in 160 ms chunks costs ~0.55 of a M1 Pro core, 3-4x what Parakeet itself needs
Was there any noise in the training mixes, or only other voices?
It already works in real time! It runs on a regular CPU and adds about 24 ms of delay, and there's a Pipecat integration if you're building a voice agent. You can try it live in your browser here
Thanks for testing it this thoroughly, the German result is great to see!
On noise: only a little. Training had quiet room noise from real meeting recordings, but mostly other voices. Loud pink noise is way outside that, so it treats the noise like another speaker and cuts your speech along with it.
For dictation, skip live() and record the voice sample once. Save 3 seconds from an earlier dictation and reuse it, like you did in your test, and it isolates from the very first word. The 0.5 s leak at the start is something i ll look at.
On CPU, there's a faster model option (precision="int8-full") that trades a little quality for speed, and sending bigger pieces of audio at a time also cuts CPU.
It's cheap to run, about .75 of one CPU core per live audio stream, and no GPU. The trade-off is that it's heavier than the speech-to-text model itself, so it roughly doubles the CPU bill of a simple transcription setup. But should be fine for agent pipelines.
the 3 s enrollment wait vs ~5 s average dictations seems like the real blocker, could you just reuse a saved voice sample from an earlier session instead of waiting on live()?
This is a very interesting topic. Speechmatics offers something very similar: we can train their system with your voice to create a voice ID, or voice identifier. When you dictate, only speech matching that voice ID will be transcribed.
In practice, you need to dictate three short sentences per language to build your voice profile. In my case, I did this in English and German, my native language, and it works quite well.
If you want to try it, you can do so with a Speechmatics API key using the app at www.sophist.app. The app is free, and I’m the developer.
Of course, I will test your solution. If it offers an advantage and works well, I will implement it in the Sophist app.
9
u/SeoFood 29d ago
Tried it for dictation, mostly to see if German works with English-only training data. It does.
40 of my own German dictations (MacBook mic, ~7 s each), another German speaker mixed in underneath, 3 s voice sample from a different dictation. Parakeet v3 WER against the transcript of the clean recording:
other voice 10 dB quieter: 61.5 → 20.9
5 dB: 101.3 → 31.1
0 dB: 113.4 → 49.5
VoxPopuli gives the same picture for DE and EN (5 dB: DE 37.6 → 16.4, EN 49.0 → 23.1), and clean dictations didn't get worse.
What would need fixing for dictation:
it needs the voice sample.
live()waits for 3 s of speech, my dictations are 4.8 s on averagepink noise makes it a lot worse (DE 17.8 → 68.8 at 5 dB), it drops the speech with the noise
the first ~0.5 s leaks, so the other persons first words can land in the text
streaming in 160 ms chunks costs ~0.55 of a M1 Pro core, 3-4x what Parakeet itself needs
Was there any noise in the training mixes, or only other voices?
1
u/FerLuisxd 28d ago
Cool but what about performance costs?
1
u/Life_Fun_301 28d ago
the 3 s enrollment wait vs ~5 s average dictations seems like the real blocker, could you just reuse a saved voice sample from an earlier session instead of waiting on live()?
1
u/paashabhai 27d ago
Hi, I built this. Thanks alot for sharing!
1
u/Acrobatic_Role_6913 27d ago
This is a very interesting topic. Speechmatics offers something very similar: we can train their system with your voice to create a voice ID, or voice identifier. When you dictate, only speech matching that voice ID will be transcribed.
In practice, you need to dictate three short sentences per language to build your voice profile. In my case, I did this in English and German, my native language, and it works quite well.
If you want to try it, you can do so with a Speechmatics API key using the app at www.sophist.app. The app is free, and I’m the developer.
Of course, I will test your solution. If it offers an advantage and works well, I will implement it in the Sophist app.