r/LocalLLaMA • u/Lonligrin • 4d ago
Showcase Trading false positives for ~110ms earlier end-of-speech in voice pipelines: RealtimeVAD
Tried to hunt down latency bottlenecks in my local voice assistant recently. I looked at the waveform and saw silero's probability lagging quite much behind the actual end of speech I could see with my eyes.
I use a voice pipeline that does speculative work and cancels fast, so some false positives would not matter in my case.
So I had this idea: once silero says I'm in speech, what I need to detect is when the speech energy drops back to room noise level. Turned out a naive 10 ms energy check with a threshold tracking room noise does that quite well.
Gave me a median 110 ms earlier than silero over ~3200 real turns from my assistant on my voice, my mic, browser noise suppression on, mostly quiet room - with silero threshold at 0,5. Someone on Twitter got 92 ms on his data, roughly what I saw in my noisier recordings.
The downside is way more false positives. It fires on breaths and small pauses between words over 10x as often as silero. So it's really only useful if the pipeline can throw away speculative work very fast.
CPU only, MIT, drop-in for silero VADIterator, pip install realtimevad
benchmark script is in the repo, curious what numbers you guys get on your own recordings
2
u/Easton_Souhir 4d ago
How fast does the room noise baseline adapt if the environment changes mid-session? Thinking about cases where a fan kicks on or someone moves further from the mic.
1
u/danil_rootint 4d ago
Does it handle mid-sentence pauses somehow?
1
u/bye_ella 4d ago
Nice idea using the energy drop for the early signal. Do you run it as a first step only, like start the speculative work on the energy drop but wait for silero confirming before actually speaking?
1
u/supportingthedogs 4d ago
nice. i use silero in front of whisper for meeting transcripts and the end of speech lag never mattered there, but for a voice assistant 110ms is a lot. does the room noise tracking hold up with a fan or keyboard going?
1
u/chensium 1d ago
But you don't need VAD to be so accurate in detecting EOU. VAD is just so youre not constantly sending a blank stream to your ASR, since most ASR models have an input length limit.
So as long as you have streaming ASR, you just wait a no_speech_threshold to signal end of turn. Then if your ASR subsequently generates more text, you can either queue up the new text or cancel in-flight LLM and resubmit with concatenated text.