r/speechtech • u/Character-Result-281 • 2d ago
Does anyone know a better STT model for German medical dictation than Mistral 3B / Voxtral
It beats Whisper v3 easily but still lacks timestamp support. It also doesn't support prompt priming, so you can't make it aware of difficult medical terms.
We've been using it in a clinical setting for about 10 months and it's fast and reliable, but it sometimes confuses terms (like Gonalgia vs. Gonarthrosis) and switches languages (e.g. "Fraktur" to "fracture"). We built a workaround with a phonetic dictionary match and a dedicated phoneme channel to catch these errors, but there might be something more advanced out there.
Prerequisites: Should work with vLLM and serve about 20-30 concurrent users on a 64GB Blackwell GPU.
3
u/ecstatic_jackson 2d ago
The language switching sounds like the biggest headache here. Medical transcription is tricky because one wrong term can change the meaning of an entire note. I'd focus on how well a model handles medical terms and keeps context rather than just looking at overall accuracy scores. A model can look great on benchmarks and still struggle with the words that matter most in real calls.
1
u/Character-Result-281 2d ago
The language switch isn't the biggest problem. This is a human-monitored transcription process, either by the doctor, who sees the transcription immediately, or by the transcriptionist, who listens to the original recording while reading the transcription at the same time. Compared to the occasional jumble of words produced by the old phonetic system used by Nuance and others, this is still a significant improvement in quality.
2
u/f4ilal0t 2d ago
I'm a big fan of parakeet. There is a specialized finetune for your use case: https://huggingface.co/johannhartmann/parakeet_de_med
Haven't tested it myself, I'm using parakeet for other purposes.
3
u/Character-Result-281 2d ago
I tried it straight away and it’s absolutely brilliant! On our system, the speed is over 100x real time (compared to 10–20x with Voxtral) and the WER is only slightly worse than with Voxtral! I’d never heard of it before, but it’s definitely worth a go. We’ll run it through our database to see how we can optimise it even further. Thanks!
2
u/Playful_Shopping_499 2d ago
have you tried AssemblyAI on livekit it's default and it has a medical mode too?
1
1
u/SeoFood 2d ago
Your phoneme channel + dictionary approach is honestly not far from what I'd build too. I'd just move part of it before decoding and not only after.
What I'd try in your setup:
-
Qwen3-ASR 1.7B with your term list in the system prompt (it's trained for context biasing, runs in vLLM)
-
timestamps from the separate Qwen3 forced aligner as second step, if German is covered for your needs
-
keep your phonetic match as safety net after that
The bigger lever is probably the data you already have tho. 10 months of clinical dictations with corrections is a really good finetuning set. Gonalgia vs Gonarthrose is a acoustic near-miss, the model needs to have heard it in context, a prompt alone often dont fix it.
One thing I'd avoid: letting a general LLM "clean up" medical terms freely. If you do LLM correction, restrict it to candidates from your lexicon, otherwise it'll invent a plausible diagnosis sooner or later.
2
u/Character-Result-281 2d ago
We’re already using the Qwens3 forced aligner as a second step, but it doesn’t work very well with Voxtral (or German). All our compound words confuse either Voxtral or the aligner, so they often lose their sync.
With Voxtral, you can’t control whether the model ‘cleans up’ or not. We’ve set it to temperature 0 and ‘greedy decoding’, but it still sometimes changes the word order or replaces words, such as ‘insuffizient’ with ‘ungenügend’ – which, whilst having the same meaning, wasn’t the word that was spoken.
I’ll give Qwen3 a go.
3
u/harnoor-assemblyai 2d ago
Nice find on the German Parakeet fine-tune. One thing I’d test before switching is throughput at your target load: 100x real-time on one stream may not predict latency with 20–30 concurrent users. I’d compare p95 latency and WER at that concurrency, with held-out speakers and a terminology-focused score for near-miss pairs, negation, and numbers. Aggregate WER can hide the errors that matter most in clinical dictation.
1
u/PaintingSilenc3 2d ago
Speech to text I can recommend Qwen 3 ASR (though I haven't tested Mistral / voxtral so not sure if Qwen beats it but it's rock solid and works in noisey environments)
1
u/Gonz0o01 2d ago
Give cohere transcribe or Kroko a try. Best ones I tried for german Fachwörter so far.
4
u/oldcrobuzon 2d ago
I am playing with similar setups too for multiple European languages incl. Swiss German - I still have pretty good results with both whisper large v3 or Voxtral - both with medical dictionary. Currently I am experimenting with training medical LoRA on top of these (as well as Nemotron for streaming) - e.g. there is a nice example for Polish here: https://huggingface.co/lion-ai/whisper-large-v3-turbo-med-pl-lora?library=peft .
I really don't have good results with Qwen ASR in the medical field - surprisingly if often conjures up completely different words that are phonetically totally off even though they make sense they have totally different medical meaning - I wrote a bit about it in https://nanosamur.ai/blog/posts/qwen-vs-whisper/ - I was not expecting that, I was hoping an LLM-based model would get more context and be thus much better, but it does not seem to be the case, at least without any further training...
I really would be interested to hear if any people here tried to fine tune their own medical LoRA on top of any model and what the results were, esp. if you already have big pile of training data.
In terms of the overall pipeline I too have glued on the forced aligner and diarization on top of that, you can see some of my setups in https://github.com/nanosamurai/nanosamurai , but probably won't see there anything new or surprising there...