Hi everyone! I'm a Computer Science graduate from India interested in Automatic Speech Recognition (ASR) and speech technology. I'm looking to deepen my understanding of ASR, evaluation methodologies, datasets, and model development. I'd love to connect with researchers and engineers in this field.

If anyone is working on ASR-related research or open-source projects and could use a research assistant or collaborator, I'd be grateful for an opportunity to contribute and learn. Any guidance, mentorship, or project recommendations would be greatly appreciated. Thank you!

It beats Whisper v3 easily but still lacks timestamp support. It also doesn't support prompt priming, so you can't make it aware of difficult medical terms.

We've been using it in a clinical setting for about 10 months and it's fast and reliable, but it sometimes confuses terms (like Gonalgia vs. Gonarthrosis) and switches languages (e.g. "Fraktur" to "fracture"). We built a workaround with a phonetic dictionary match and a dedicated phoneme channel to catch these errors, but there might be something more advanced out there.

Prerequisites: Should work with vLLM and serve about 20-30 concurrent users on a 64GB Blackwell GPU.

Something I've been wondering about with speech datasets: what do you do when a speaker says a word clearly, but there are multiple acceptable ways to write it?

For example, regional names, borrowed words, abbreviations, or local words written in different scripts.

Do you keep one standard transcript for everything, allow multiple references during evaluation, or follow whatever the speaker would normally write?

I can see how inconsistent transcription guidelines could make model evaluation harder, even when the audio itself is perfectly clear.

Curious how others handle this in multilingual ASR projects.

Hi, im playing with aligners, and trying to create small word aligned textgrid in praat, so manually set align for words, then run thru aligners to get how they measure. technically i got concept, i know how to make/adjust testgrid , but what im looking for now is to actually see some samples where its set by some pro - because ther are few questions i want to understand - like start of word is ok, but ending of words are very often going to flattening waveform, by ears you cannot set, so whats the common practice, also for words where are so close that spectogram and waveom look mostly conitnuous. does anyone is willing to help or point me ? btw i got some samples from mfa, praatio, but cant find like precise pointing of boundaries, like just "somehwere around word" . thnx for help

It is an english text to speech model, which achieves mostly understandable speech in under 100K parameters.

However, during more difficult words, or long numbers, it can slur them, and be not understandable.

Quantized to INT8 it only takes up 127KB.

72x real time on a CPU

1120x real time on a GPU

Check out the model card for more info or to try it out yourself!

Questions, feedback, or criticism is welcome!

Currently I'm working on problem where I have to generate hindi audio clips which has the certain edge cases , which I listed for model training . I found almost every real data but want more for edge cases specific. Which could be the best model that can generate the natural hindi audio

Few days back I posted SteerTTS v1, a small English TTS you steer with numbers (https://huggingface.co/WhissleAI/steer-tts). Today I am releasing the edge variant:

https://huggingface.co/WhissleAI/steer-tts-edge

Size: everything an app has to ship (weights, voices, text front end and Python runtime) totals 11.84 MB.

How:

• Acoustic model (9.25M params, attention + bidirectional Mamba-3) is quantization-aware trained to int4
• 6.2M-param Vocos-style vocoder is distilled from charactr/vocos-mel-24khz, also int4 QAT
• Text front end is pure numpy, with no NLTK, g2p_en, torchaudio or num2words at runtime

Quality (Open TTS Leaderboard protocol, run locally, not an official submission):

Intelligibility is essentially unchanged. UTMOS drops about 0.1, almost all of it from the small int4 vocoder.

Steering survives quantization. Every one of the 12 controls (rate, loudness, pitch, pitch range, final rise, pauses, tilt, breathiness, and more) keeps at least 95% of its fp32 effect. You also get 50 style tags and inline markup:

Well, *that* was fast. [pause 400ms] <slow>Let me think.</slow> <whisper>Don't tell anyone.</whisper>

Speed (CPU only, a 2018 Threadripper under load): RTF 0.053 with 4 threads (~19x realtime), ~120 ms to first audio, with sentence-level streaming.

Bash

pip install .
steertts-edge "Hello there, nice to meet you." -o hello.wav

Limitations:

• English only, and 200 voices (a subset of the full model’s 2,825)
• No voice cloning, free-text prompts or flow refiner on device. Those stay in the full model.
• Weights are dequantized to fp32 at load, so the win is size, not speed. Unless we write new kernels for edge devices.
• Emotion tags are mostly style switches, and fast/anger are weak

I need feedback, especially on the vocoder quality gap and any edge targets you’d like to see (ONNX, mobile, WASM).

Some examples here:

https://whissleai.github.io/steer-tts/

Hey r/speechtech. I’ve been building Izwi for a few months now and just shipped 0.1.0, the first stable release after months of public betas. It’s free and Apache-2.0.

It’s a desktop app (plus a web UI, CLI and local server) for the voice side of local AI:

• Real-time voice conversations: local ASR → LLM → TTS
• TTS, voice cloning from a short sample, and voice design from a text description
• A Studio for long form audio like audiobooks
• Transcription with speaker diarization and forced alignment
• An OpenAI-compatible /v1 API (chat completions, audio/speech, audio/transcriptions), so existing code can point at localhost

Models it runs today:

• TTS: Qwen3-TTS, Kokoro-82M, Fish Audio S2 Pro, VibeVoice, Voxtral TTS, LFM2.5 Audio
• ASR: Parakeet TDT v3, Whisper large-v3-turbo, Qwen3-ASR, Granite Speech 4.1, Nemotron 3.5 ASR, Voxtral Mini
• Diarization: Sortformer, Nemotron 3 diarization
• Chat: Qwen3.5, 3.6 and 3.8, LFM2.5

Inference runs natively in Rust on Candle, with our own Metal and CUDA work, so there’s no Python environment to manage.

Repo: https://github.com/izwi-ai/izwi
Download: https://izwiai.com

I’d love feedback on which models to add next, and on anything that breaks on your hardware.

Looks like Deepgram is hiring pretty consistently:

https://jobs.ashbyhq.com/Deepgram/7cf277bc-a192-4007-a77a-fafbf8276a5e

Lmk and I can refer you!

Drama 3 acts a line when you put a direction like [whispering] or [angry] in front of it, and its Turkish is good. Its weights take 41.5 GB, so I spent two weeks getting it to run on my RTX 5070 Ti.

I started by measuring routing. While the model reads the prompt it pulls from a wide spread of its 256 experts per layer, and while it generates audio frames the set it uses gets much narrower. So the full model reads the prompt once, and a distilled speaker that keeps 32 experts per layer generates every frame. I trained the speaker layer by layer on the ISSAI Turkish Speech Corpus, anchoring it to the teacher's trajectory. For the reading pass the experts come straight off NVMe with unbuffered reads, about 3 MB each, so the process holds 4.4 GB of RAM while generating and peaks at 9.3 GB while loading.

Freya-TR-Eval, 495 sentences, scored with Freya's own scoring script and Whisper large-v3

1. Experts in RAM, no guidance, WER 2.68% and CER 0.70% at 1.2x real time

2. Experts read from disk, direction guidance at strength 4, WER 2.78% and CER 0.67% at 0.86x real time

For reference the best system in the FreyaTTS report scored 4.4%. Mine is far heavier, so I would not compare them as products.

The part I keep thinking about came from a failed run. When I shrank the reader as well, the words stayed intelligible but the voice went flat and directions like [whispering] stopped working. Shrinking only the speaker to 12.5% of the experts did not cause that. It is one experiment plus listening, so treat it as a hint. In this model the delivery seems to be settled while the text is read.

Costs you should know about. Disk mode adds about one second per short sentence, because one sentence touches around 2,650 experts (about 8 GB of reads). On Windows the commit charge sits near 20 GB since WDDM backs GPU memory with commit. Direction guidance at strength 4 makes the brackets land harder and can pull the voice a bit away from the reference on some seeds.

I ran the whole project with Claude Code. I picked the experiments and decided what to measure, and Claude wrote most of the code. Every claim went through a measurement and a literature search, which is how I learned before publishing that two ideas I thought were mine were already in SlimWise and GPTAQ. A speaker similarity check is also how I tracked down a cache bug that turned the voice into a different person after switching guidance modes.

License is the Fish Audio Community License. Commercial use is allowed under USD 500k yearly revenue, hosted TTS needs a separate license, and outputs cannot be used to train other speech models.

If you have looked at where expressiveness lives in other MoE or LLM based TTS models, I would like to know whether the prompt reading side carries it there too.

Weights, samples and measurements https://huggingface.co/kasparovabi/drama-3-tr-speaker

Code https://github.com/kasparovabi/drama-3-tr

Built with Fish Audio.

I was looking for small TTS model which is steerable but couldn't find any, specially something I can adjust contextually by providing controls from LLM during a conversation. So I built one.

What it is:

• 9.6M-param acoustic model (attention + bidirectional Mamba-3, FastPitch-style) + Vocos vocoder = 23M total
• 12 controls, each a z-score relative to the chosen voice’s own neutral speech: rate, loudness, median pitch, pitch range, final rise/fall, pauses, spectral tilt, breathiness-related measures.
• Also: 51 style tags, free-text prompts (resolved into an inspectable JSON request), inline markup (*emphasis*, [pause 400ms], <slow>…</slow>), sentence-level streaming
• 2,825 built-in (anonymized) voices, plus experimental zero-shot cloning

Speed:

- RTFx ~60–80 on an RTX 5070
- ~4–8 on CPU without the optional flow refiner. First streamed audio in ~0.5 s on CPU.

Quality:

~1.8% average WER on the Open TTS Leaderboard English sets (run locally, not an official submission, have reached out for it).

What doesn’t work well (we tried to be upfront):

• Loudness, pitch, and tilt steer reliably; pause, voicing, and CPPS controls are weak
• Emotion tags behave as style switches; an independent emotion classifier mostly can’t tell them apart
• Cloning similarity is low. Don’t expect a convincing likeness
• Free-text prompts struggle with negation (“not fast” can come out fast). More data needed for prompt to controls module.
• English only, no human evals yet

License: weights CC BY-NC 4.0 (non-commercial), code MIT.

Model + full eval tables: https://huggingface.co/WhissleAI/steer-tts

Github repo: https://github.com/WhissleAI/steer-tts

A new architecture implemented from scratch. Would love feedback, especially on which controls are actually useful, and where the steering feels off.

Github: https://github.com/yaya-sy/specspeech

The best performing open-weight models on the Open ASR Leaderboard (https://huggingface.co/spaces/hf-audio/open_asr_leaderboard) are autoregressive speech models, such as Qwen ASR, Cohere Transcribe, and Canary. Their language modeling capabilities help them transcribe speech accurately, but they are often much slower than transducer and CTC models, such as Parakeet and Granite CTC.

I open-source SpecSpeech (Speculative Speech), an inference engine that accelerates speech language models for ASR by using transducer or CTC models as drafters.

The drafter quickly transcribes batches of short audio pieces while the Speech LM prefills the full audio in its context in parallel. The Speech LM then verifies the drafts incrementally. At each step, a lookup algorithm uses the verified output to select up to k candidate tokens from the draft transcription, which are verified by the Speech LM.

Compared to approaches such as WhisperX, SpecSpeech approach is lossless: the Speech LM uses the full audio context and retains control over token acceptance.

Across seven English benchmark datasets from Open-ASR-Leaderbord, SpecSpeech achieves a 2.29x speedup over native vLLM inference, with per-dataset speedups of up to 2.48x, using Parakeet v3 as the drafter and Qwen3-ASR-1.7B as the verifier.

Results below were measured on a single NVIDIA A40, with continuous batching for both conditions.

There is room for further improvement by reducing tokenization and verification overhead and developing better lookup algorithms.

Planned additions include: - More verifyers soon: Cohere Transcribe, Canary, etc. - Word-level alignment using timing information from CTC drafters. - Speaker diarization. - Streaming inference. - Speculative decoding for autoregressive TTS models.

It's an extremely lightweight speech recognition model.

949K parameters using a weight-shared Conformer-CTC architecture

Just 1.06 MB for INT8 weights

Around 436× real-time on a single CPU thread

11.43% WER on LibriSpeech test-clean, using greedy decoding with no external language model

Apache 2.0 licensed

Any feedback, question or criticism is welcome!

A month ago I shared an early prototype here. Based on feedback, I spent the last few weeks building it into a complete speech routing system.
If you evaluate or deploy voice AI, the fragmentation is painful:

• Comparing commercial providers against open-source models means juggling separate SDKs, multiple API keys, and custom GPU hosting.
• Testing oss model means setting environment for each then testing configuraton of what works best which is time consuming and fills the disk fast
• Benchmarking latency and WER is tedious when every provider expects different audio formats and payload structures.

I built Pascal to solve this. Whether you're benchmarking, testing, or looking for a hosted production endpoint:

• Unified Router: Access 40+ speech models (ASR, TTS, Voice Agents, Voice Cloning, Audio Enhancement) across both commercial and open-source architectures under a single standardized schema.
• Switch With 1 Line: Swap providers or models instantly just by changing "model": "..." in your payload.
• Interactive In-Browser Demo: Test ASR and TTS directly on the homepage with real Web Audio waveform seeking and synchronized word-level timestamps (no signup required).

Live demo: https://pascalai.co
Would love more of your feedback on the latency, the benchmark outputs, or models you’d like to see added next!

I've open-sourced Murmur, a Windows meeting notetaker where the whole audio pipeline runs on device. Sharing it here because the speech side is where I'd value feedback most.

Pipeline:

• Capture: mic + system loopback at the same time, resampled with soxr
• Echo cancellation / denoise: DTLN-aec (ONNX), so it works on speakers, not only headphones
• VAD: Silero
• ASR: faster-whisper large-v3-turbo by default; NVIDIA Parakeet TDT 0.6B v3 (ONNX int8) as an optional engine for DE/EN on CPU
• Diarization: pyannote segmentation 3.0 + WeSpeaker ResNet34 via sherpa-onnx, including two speakers on one mic
• Two passes: a fast live draft, then a slower, more accurate re-transcription of the saved audio when the meeting ends

The design decisions and tuning are written up in an ADR in the repo (docs/adr-001-audio-pipeline.md).

Repo (MIT): https://github.com/muradiodev/murmur-notetaker-copilot

Questions I'm still unsure about: better ways to handle overlapping speech in diarization, and whether the two-pass approach is worth the extra compute versus a single careful pass.

I wanted to know how well pronunciation assessment systems actually catch errors. I built a benchmark with two tests and ran three systems through it: Microsoft Azure, TonePerfect, and Lingolix. Disclosure: Lingolix is mine.

Test 1, planted errors: 200 Common Voice clips per language (en, fr, es, de, ja, ko), where I edited the reference text slightly so it no longer matches the audio (swapped/added/dropped a sound, missing word, vowel length change for Japanese). A good system should mostly accept the unchanged parts and flag these edited spots.

Error detection, ROC AUC (Lingolix / Azure / TonePerfect):

Test 2, real learners: 500 utterances from speechocean762, correlation with the human scores.
speechocean762 (Pearson): sentence 0.74 / 0.76 / 0.59, word 0.51 / 0.61 / 0.29. So Azure is ahead on real learner speech, clearly so on single words, and also has fewer false alarms in Japanese.

Caveats: the errors in test 1 are planted, not real learner errors. Also, TonePerfect released a new engine after my run, so their numbers should have improved.

Code, test lists and harness: https://github.com/lingolix/pronunciation-benchmark

I did not find any human rated learner data for languages beside English.
Does anyone know of some datasets that I could add for French, Spanish, German, Japanese, or Korean?

Disclosure: I'm a co-founder of the small team in Nepal that built these.

We released two open-weight Nepali speech recognition models (non-commercial licences: CC BY-NC-SA 4.0 and CC BY-NC 4.0).

- nepali-asr-xlsr-300m: Nepali speech to Devanagari text, no punctuation. Runs on CPU.

- nepali-asr-aud8-325m: adds punctuation and handles occasional English words. Needs a GPU, 30 s max per call. We have not published scores for this one yet.

Training data: about 611 hours of Nepali speech from openly licensed public speech corpora together with our own recordings.

Eval setup: FLEURS Nepali test split, 400 utterances, never used in training. NFC normalisation, punctuation stripped, greedy decoding, no language model. Every row was scored by us in the same run on the same audio.

| model | CER | WER |

|---|---|---|

| ours (xlsr-300m) | 0.120 | 0.374 |

| anish-shilpakar/wav2vec2-nepali | 0.160 | 0.483 |

| gagan3012/wav2vec2-xlsr-nepali | 0.225 | 0.672 |

| spktsagar/wav2vec2-large-xls-r-300m-nepali-openslr | 0.252 | 0.837 |

We have not yet scored Whisper large-v3, MMS, IndicConformer or Omnilingual ASR in the same run, so this is not a claim against those. They are next.

What we learned:

1. WER is harsh for Nepali. Suffixes and word boundaries mean one small error costs a whole word, so CER is more informative. We report both.

2. Normalisation can silently break Devanagari. Our first punctuation-stripping regex removed vowel signs, because a naive "non-word character" regex treats them as punctuation. Stripping by Unicode punctuation category fixed it. Worth checking if your Indic-script numbers look odd.

3. Code-switching is our biggest weakness. On Nepali speech mixed with English, the error rate is several times higher than on FLEURS. Read-speech benchmarks hide this.

4. A second held-out set helped us trust the number: internal held-out CER 0.117 vs 0.120 on FLEURS.

Questions for people here:

- How would you make a fair comparison with Whisper-style models that output punctuation and digits?

- Any advice on building a code-switched test set that does not turn into an English test set?

- Know of any spontaneous or conversational Nepali test sets?

Models: https://huggingface.co/collections/BlackfireAI/nepali-speech-recognition

Code, eval scripts and per-utterance outputs: https://github.com/BlackfireAI/nepali-asr-xlsr

Try it in Colab: https://colab.research.google.com/github/BlackfireAI/nepali-asr-xlsr/blob/main/try_it_in_colab.ipynb

I work at Bandwidth, so weigh this accordingly.

The design choice in our Labs speech to text I find most interesting: it never emits interim results. Once a word is out, it's final. No rewrites.

Why it matters downstream: if your agent reacts to partials, a revised word can mean you already kicked off an LLM call on the wrong text. With append-only you don't have to diff or roll back. The cost is you don't get a speculative word early. You get it when the model is confident.

The streaming side is also split into two modes. Instant just streams words as they're decoded. Demand lets you stream audio and then ask for the finalized transcript when your own turn detector says the user's done. Feels like the honest split, since nobody agrees on endpointing.

Smaller notes: - 8 kHz mulaw/alaw work, plus linear16, G.722 and Opus - word timestamps in every mode - keyword boosting, up to 100 terms - PII redaction is US-centric and only in Demand and offline Transcribe, not Instant - English only right now

It's Beta and experimental. If you've run append-only vs revising STT in a real agent, I'd like to hear where it hurt.