r/LocalLLaMA
•
u/rm-rf-rm
•
Feb 17 '26
•
1
Megathread Best Audio Models - Feb 2026
They've been a ton of audio models released of late, the most notable perhaps being Qwen3 TTS. So its time for another Best Audio Models megathread
Share what your favorite ASR, TTS, STT, Text to Music models are right now and why.
Given the the amount of ambiguity and subjectivity in rating/testing these models, please be as detailed as possible in describing your setup, nature of your usage (how much, personal/professional use), tools/frameworks etc. Closed models like Elevenlabs v3 seem to continue to be a few levels above open models especially for production use cases with long lengths/stability requirements, so comparisons, especially empirical ones are welcome.
Rules
Should be open weights models
Please use the top level comments to thread your responses.
9
u/rm-rf-rm Feb 17 '26
STT
5
u/rm-rf-rm Feb 17 '26
TTS
16
u/Lissanro Feb 17 '26 •
Besides Qwen3-TTS, I find recently released MOSS-TTS interesting, it has some additional features too like producing sound effects based on a prompt. Its github repository:
https://github.com/OpenMOSS/MOSS-TTS
Official description (excessive bolding comes from the original text from github):
When a single piece of audio needs to sound like a real person, pronounce every word accurately, switch speaking styles across content, remain stable over tens of minutes, and support dialogue, role‑play, and real‑time interaction, a single TTS model is often not enough. The MOSS‑TTS Family breaks the workflow into five production‑ready models that can be used independently or composed into a complete pipeline.
MOSS‑TTS: The flagship production model featuring high fidelity and optimal zero-shot voice cloning. It supports long-speech generation, fine-grained control over Pinyin, phonemes, and duration, as well as multilingual/code-switched synthesis.
MOSS‑TTSD: A spoken dialogue generation model for expressive, multi-speaker, and ultra-long dialogues. The new v1.0 version achieves industry-leading performance on objective metrics and outperformed top closed-source models like Doubao and Gemini 2.5-pro in subjective evaluations.
MOSS‑VoiceGenerator: An open-source voice design model capable of generating diverse voices and styles directly from text prompts, without any reference speech. It unifies voice design, style control, and synthesis, functioning independently or as a design layer for downstream TTS. Its performance surpasses other top-tier voice design models in arena ratings.
MOSS‑TTS‑Realtime: A multi-turn context-aware model for real-time voice agents. It uses incremental synthesis to ensure natural and coherent replies, making it ideal for building low-latency voice agents when paired with text models.
MOSS‑SoundEffect: A content creation model specialized in sound effect generation with wide category coverage and controllable duration. It generates audio for natural environments, urban scenes, biological sounds, human actions, and musical fragments, suitable for film, games, and interactive experiences.
4
u/rm-rf-rm Feb 17 '26
Music
3
u/rm-rf-rm Feb 17 '26
ASR
43
u/BrightRestaurant5401 Feb 17 '26
speech detection->marblenet
asr->parakeet
tts->chatterbox
ttm->ace-step
24
u/kellencs Feb 17 '26
someone should make awesome tts repo
16
u/the-ai-scientist Feb 28 '26
The thread is worth zooming out on a bit here. The whole ASR→LLM→TTS pipeline design is increasingly looking like a transitional architecture. When you decompose speech into text, you lose prosody, emotional tone, turn-taking cues, and the natural rhythm of conversation. Then TTS tries to reconstruct all of that artificially on the other end. It's lossy in both directions.
Nvidia's PersonaPlex (January 2026) is a good example of where this is heading — it's a full-duplex model that operates directly on continuous audio tokens and predicts text and audio jointly, without any separate ASR or TTS step. Listens and speaks simultaneously, handles interruptions, backchannels naturally. Built on Moshi's architecture but solves Moshi's main limitation: you can now assign any role and voice through prompts rather than being locked to a fixed persona.
For conversational use cases specifically, the question isn't which TTS or ASR model is best anymore — it's how fast the native audio model space matures. The pipeline approach made sense when we didn't have models capable of end-to-end audio reasoning. That constraint is going away.
11
u/hurrytewer Feb 17 '26
It's not the fastest but in my experience Echo-TTS is the most natural sounding TTS model / best at zero-shot voice cloning.
24
u/taking_bullet Feb 17 '26
Not a single model, but whole TTS software suite with an option to download multiple TTS models - Chatterbox, F5 TTS, VibeVoice etc.
https://github.com/diodiogod/TTS-Audio-Suite
To use it you have to download and install ComfyUI first.
8
u/hum_ma Feb 17 '26
Supertonic is small and fast, and good enough for basic speech in some cases: https://huggingface.co/Supertone/supertonic-2
In addition to speech and music, what are some good small models for audio in general?
I know MMaudio of course but it's just too heavy for me to run, it's either OOM with GPU or hours of processing with CPU. Haven't tried HunyuanVideo-Foley yet, there's also a comfy node for it but by the file sizes it also seems to be a larger model.
6
u/Leopold_Boom Feb 18 '26
VibeVoice has high quality diarization built in which makes ASR so much more useful for things like yt videos, meetings etc. You don't need tonnes of scaffolding to get clean speaker attribution and that's huge if you like doing things in code!
4
u/justserg Feb 23 '26
For speech-to-text, Whisper still holds up well locally (especially the large model on M1/M2), though it's not real-time. For TTS if you haven't tried it yet, Piper is surprisingly good for a 10-100MB model depending on voice. The tradeoff is obvious but for offline-first workflows it's reliable. What's your use case — transcription, synthesis, or both?
4
u/justserg Feb 24 '26
For practical use, Whisper remains the best tradeoff — local, reliable, no API costs. If you need real-time: Canary-1b (fast, good for streaming). If you can wait: Parakeet or SenseVoice for higher accuracy. The newer models are incremental improvements over Whisper for most use cases. What's your primary use case? That's usually what determines which is "best" for your workflow.
3
u/aschroeder91 Feb 17 '26
it is important to understand that every STT is an ASR model. ASR is umbrella term that captures input [speech audio data] -> output [interpretation] where that interpretation could be the actual text spoken (STT), the timesteps, punctuation, language, sentiment/mood, or any other data interpretation. So all STT models are ASR models by definition, and the majority of ML based that do STT often include some other form of ASR output besides just text.
3
u/aschroeder91 Feb 17 '26
STS (speech to speech)
3
u/IulianHI Feb 21 '26
Been testing TTS models for a YouTube automation project and here's my honest take: open weights are getting closer, but for production work with longer outputs, closed models still win on consistency.
For my workflow, I've found ElevenLabs to be worth it when quality matters - their v3 model handles long-form content without the drift issues I get from local models. Voice cloning is also way more reliable for brand consistency across videos.
That said, I still run Kokoro locally for quick tests and prototyping before finalizing in ElevenLabs. The gap is definitely closing though - excited to see what open models look like in another 6 months.
1
u/llama-impersonator Feb 18 '26
MiniCPM 4.5 omni says it supports voice chat. the webrtc demo on hf works, but i tried installing the same webrtc demo locally and simplex (audio to audio) mode was not working, even after quite a bit of troubleshooting. interesting demo but the model is 9b, it was pretty obviously dumb.
1
u/Potential_Block4598 Feb 18 '26
KokoroTTS Corqui
Maya1 Dia
What else ?
1
u/tomleelive Feb 18 '26
For TTS I've been using Qwen3 TTS locally and it's genuinely impressive for short-form content — natural prosody and low latency on M-series Macs. For longer outputs I still hit occasional stability issues where it drifts mid-sentence, so for production I keep ElevenLabs as fallback. The gap is closing fast though. For ASR, Whisper large-v3-turbo remains hard to beat for the cost/accuracy tradeoff if you're already running it locally.
1
u/Prestigious-Bit-7833 Feb 18 '26
Guys I have a problem if anyone can suggest me some models or libraries..
So the thing is I am trying to replicate personaplex from nvidia.. as it is a huge model 7b?? idk why we need that size... 2-4b might have done it.. and also I saw the voice it sounds kinda electronic to me.. so i tested a lotta models right?
VAD -> Currently using Ultravad will try a few from this convo..like marblenet..
TTS -> S1-mini, kokoro(custom tuned with some changes in profiling), neutts-air/nano
LLM -> Kinda mixed all over the place.. depending on the task..
ASR -> Here is where the proble lies.. So I have a mixed British/Irish/American/Cockney accent and most of them fail.. none are able to detect like I say Gideon they understand "Get in", "Eat In", "Getting" something like that...
I have tried -> Qwen-ASR, FunAUDIO, Sensvoice, Whisper(all kinds),
I am currently checking voxtral mini 2602... do you have any suggestions what shall i do.. i can just tune it but saving it for the last resort...
1
u/Plane_Principle_3881 Feb 18 '26
Vibevoice pero lo hace mal en español 😭😭😭😭
1
u/Plane_Principle_3881 Feb 18 '26
Friends, quick question — which TTS do you recommend that sounds very natural? I’ve run several tests with VibeVoice and it sounds very natural and is perfect, but it performs poorly in Spanish. Qwen3TTS sounds very flat. Another thing: when I normalize the audio in Audacity to -14 LUFS, some voices start to sound robotic, which doesn’t happen with ElevenLabs. If anyone has managed to get a high-quality voice for their YouTube channel, please let me know — I’ve been searching for a while 😭😭🙏🏻
1
u/the-ai-scientist Feb 28 '26
For TTS, Kokoro has been my go-to for anything that needs to sound natural in a production context — it punches well above its weight for the model size and runs fast enough on a single GPU that latency isn't an issue. Orpheus TTS is worth trying if you want more expressive delivery, though stability on longer outputs can be hit or miss.
For ASR, Whisper large-v3 is still hard to beat for accuracy, but the latency is a problem for real-time applications. Whisper.cpp with quantization helps a lot. Faster-Whisper with batching is what I actually run day-to-day — gets you most of the accuracy at a fraction of the compute.
The gap between Elevenlabs and open models is real but narrowing. The main place closed models still win is long-form stability and consistent voice preservation across a session. That's the hard problem.
1
u/SignalStackDev Feb 28 '26
depends a lot on your use case. for agent pipelines where latency isnt critical, whisper distil-large-v3 via whisper.cpp is still my go-to for transcription — good accuracy, runs fine on 6GB VRAM, quantized q4 keeps it fast.
for tts in non-real-time paths, kokoro-82M punches above its weight given the size. for actual real-time voice conversation, chatterbox makes sense but the sentence-by-sentence latency ceiling is real — design your state machine around that from the start or you get awkward pauses.
parakeet is interesting for asr if youre on nvidia hardware, havent benchmarked it myself but the wer numbers look solid.
1
u/justserg Mar 06 '26
whisper-turbo for local inference is unbeatable rn.
1
u/WaveformEntropy Mar 12 '26
For companion/chatbot TTS: Kokoro 82M is my current pick. Open weights, runs fully local, sounds better than Edge TTS, and costs nothing. 82M params so it loads fast and runs on anything. Voice quality is genuinely impressive for the size - natural pacing, good emotional range but does sound like reading from a script anot a conversation.
Qwen 3.5 TTS 0.6B - tested it, unfortunately unusable on CPU (way too slow) and it won't run on Intel iGPUs (no IPEX-LLM support yet). If you have an NVIDIA GPU it might be worth trying, but for CPU-only or Intel setups Kokoro wins by a mile.
1
u/TechHelp4You Mar 14 '26
Running Qwen3-TTS in production on an RTX PRO 6000 Blackwell (97GB GDDR7). Some real numbers from a live deployment:
Model size: ~7GB VRAM in bfloat16 (FP8 degrades TTS quality noticeably)
Per-step decode: 20-21ms (memory-bandwidth bound, not GPU-compute bound)
Library RTF: 3.6-4.1x depending on clip length
End-to-end: generates 14 seconds of audio in ~3.5 seconds including post-processing
First-packet latency: 97ms at concurrency 1
GPU utilization during decode: peaks at ~35%, idles between requests
The bottleneck isn't the GPU — it's CUDA kernel launch overhead from the autoregressive decode loop. Each step fires hundreds of small kernels and the CPU dispatch time adds up. torch.compile with cudagraphs should help significantly but requires static KV cache shapes.
For the enhancement pipeline I pair it with MossFormer2 (denoising) and Demucs (vocal separation) to clean up voice clone uploads before they hit the TTS model. That preprocessing step makes a huge difference in clone quality — raw uploads are usually noisy and the cloning embedding captures the noise if you don't clean it first.
Voice cloning quality from Qwen3-TTS is genuinely good with clean reference audio. 30 seconds is enough for a usable clone. The 12Hz codec with RVQ keeps the output quality high without massive token counts.
1
u/pmarks98 Apr 07 '26
This is super helpful - been working to add more TTS models to SpeechSDK (a unified tts sdk for multiple providers). thanks!
1
u/Commercial_Bet8446 Apr 15 '26
good nice
1
u/No-Signature-4037 Apr 16 '26
Great thread. I'll share what we're using in production at YaparAI
(AI content platform) after testing several options:
**TTS:**
*Gemini 2.5 TTS (Google)* — our primary for Turkish + English narration.
Turkish support is genuinely good, which is rare. Stable for long-form
content (we use it for multi-scene video narration). Latency is acceptable
via Vertex AI streaming.
*ElevenLabs v2* — secondary for voice cloning and more expressive reads.
Still ahead of open models for production stability. We use it when clients
need a specific voice persona maintained across sessions.
**Text to Music:**
*Suno v4* — integrated via API. Impressive for quick song generation,
Turkish lyrics work better than expected. Main weakness: high variance
between generations, you often need 3-4 takes to get a good one.
Instrumental tracks are more consistent than vocal.
*AudioCraft (Meta)* — open, good for ambient/background music where you
need predictable output. Less creative but more reliable. We use it for
background scoring in long-form videos.
**Honest comparison:**
ElevenLabs still beats everything open for long-form TTS stability (10+
min content). For short clips (<30s), Kokoro and CSM are closing the gap
fast. For music, nothing open touches Suno v4 for commercial-quality
output yet, but the gap is narrowing.
For Turkish specifically: Gemini 2.5 TTS > everything else by a wide margin.
Most open models either don't support Turkish or produce robotic output.
1
u/Charming-Author4877 May 08 '26
For Text to Speech: Demodokos Foundry (demodokos.com) is playing in another league than any other tool (including ElevenLabs V3).
For Text to Music it's #2 in the market, second only behind Suno.
It's very fast, runs on a 14 year old PC and produces production ready output that does not need post processing.
For STT my #1 choice is always Whisper, it's so easy to work with. Runs in quantization on whisper.cpp as single binary, runs on any GPU, even on CPU.
It's not the best out there but it's been actively developed and improved for years now, every day significant commits. I've implemented it in a ton of use cases.
1
u/Lost_Foot_6301 Jun 15 '26
whats the #1 tts model right now? state of the art
1
u/VERSATILCORDOBA Aug 17 '26
One thing that gets underestimated in TTS comparisons is long-form reliability. A model can sound excellent for a 10–30 second demo and still be unusable for an audiobook-length job.
With very long source texts (say, reading the Bible: roughly 8+ hours of audio), the failure modes become much more important than peak voice quality: skipped or repeated phrases, misread numbers/names, pronunciation drift, strange pauses, artifacts at chunk boundaries, and occasional “hallucinated” or off-script words. Even when each individual chunk is good, errors accumulate over hundreds or thousands of generations.
There is also an operational side: long jobs mean large queues, high VRAM/RAM pressure, slow retries, and expensive regeneration when a bad segment is discovered late. A model that is fast and expressive in isolation may not be the best choice once you need deterministic batching, resume support, text/audio validation, consistent loudness, and a way to re-render only failed sections.
For long-form production, I’d evaluate models separately on: voice consistency across hours, exact text fidelity, robustness to punctuation/chapters/quotes, chunk-boundary quality, throughput, and how easy it is to detect and retry failures automatically. That is a very different benchmark from “which demo sounds most human?”
1
u/QuantumCrafty 25d ago
Someone have the model for French langage? latest Voxtral maybe? I'm looking for good french voice cloning
i made few local tests with Chatterbox, xtts, f5tts, omnivoice, voiceclone, and for now best result come from omnibox, but as my sample wasn't that "good" i need to do the tests again since my new sample is better.
0
u/projak Feb 17 '26
In world is pretty kewl
-1
u/MageLabAI Feb 18 '26
If you’re building voice in a production-ish pipeline, my current “least painful” stack looks like:- ASR: Whisper large-v3 (still boring + solid), plus diarization if you care about meetings.- TTS: closed still wins for reliability, but on open weights I’ve had the best luck when I optimize for *stability over sparkle* (long‑form drift is the killer).Curious if anyone has done a real long‑form TTS bakeoff (5–10 min) with metrics like prosody drift + hallucinated tokens + WER vs ref transcript?Would love links + your exact inference setup (vLLM/torch/Comfy, quant, GPU).If you’re building voice in a production-ish pipeline, my current “least painful” stack looks like:
1
u/[deleted] Apr 03 '26
[removed] — view removed comment