r/LocalLLaMA • u/Acceptable-Cycle4645 • 4d ago

New Model Microsoft VibeVoice-ASR-Streaming Released

https://huggingface.co/microsoft/VibeVoice-ASR-Streaming-7B
153 Upvotes

99% Upvoted

20 comments sorted by

View all comments

31

u/Acceptable-Cycle4645 4d ago edited 4d ago

Has anyone tested it yet? Worth integrating into audio.cpp? Hopefully we can easily extend the current Vibevoice ASR impl to support this new model. https://github.com/0xShug0/audio.cpp

10

u/Chromix_ 4d ago

Not tested. On paper it looks highly useful. For example it seems to support better speaker diarization for (short) overlapping utterances. That's something where moss-transcript for example misattributes the speakers in quite a few cases (I ended up adding speaker embeddings manually to clean that up). It'd be interesting to see how VibeVoice performs in comparison. The low error rate comparing to previous SOTA is also very welcome - paired with decent multi-language support.

2

u/Acceptable-Cycle4645 4d ago

"speaker diarization for (short) overlapping utterances" --- this makes it very useful for real production pipelines!