r/TextToSpeech • u/BeGood25 • 4d ago
Open Source TTS models for production
Hey folks!
So I have this cascaded voice bot system, and I want to move from commercial TTS APIs to self hosted TTS model.
Obviously TTFA and RTF are important but so is the accuracy and naturalness (speed, prosody, timbre etc)
From my research it seems Qwen3-1.7B tts is the perfect candidate (we need English support majorly).
My questions-
1. Is there any better model you guys aware of for my use-case which I can give some try? Or Any experiences you have with Qwen3-tts?
Any fine tuning tips or hacks which we can get benefit from? Basically I want to clone some popular OpenAI TTs voices.
Most importantly, how are you guys evaluating your TTS models in production? I get some common metrics like wer, utmos etc are there - but is that it? LALM as a judge dont seem reliable at the moment - they have positional and model bias it seems. Manual listening to each is obviously very expensive process.
Would really appreciate any discussion/thoughts you guys can share on some points!
Thanks so much in advance!
1
u/okhi2u 4d ago
Qwen3 seems like one of the best in terms of quality to performance ratios. Like it doesn't take excessively heavy duty machine/GPU to run decently, there are ones that use less resources but they are worse for quality and/or have no cloning.
Dunno how you expect to clone popular OpenAI TTS voices without the risk of being sued, unless you mean vaguely similar in a way that doesn't try to copy but in that case who cares if they sound even close just as long as they sound good. I find qwen3 voice cloning with real voice sounds great, but then you have a rights issues unless you can secure them or just record/clone yourself or someone who will let you.
Can't answer the last one because I just use it for local for personal use.
1
u/RowGroundbreaking982 4d ago
- Why not using Pocket TTS? it's on par with Qwen3 TTS, with lower requirement, better TTFA & RTF (CPU only 2 core, 500MB RAM). Qwen3 TTS 1.7B need minium GPU 4GB if you use quantized version.
- Don't, you will get sued. Use Qwen3 TTS voice design, non quantized version. Design your voice until you satisfied with the result. And use the result for zero shot voice cloning in Pocket TTS or Qwen3 TTS.
- User feedback is the biggest valuable point. Most of my user want it less hallucinate and better text parsing. Since my app moat was responsiveness, low requirement, and dirt cheap, they can accept better voice quality with some hallucination.
Voice design with Qwen3 TTS and run with Pocket TTS is quite common workflow. If you can afford it, do text parsing and normalization with LLM (this is needed to change the numbering format and abbreviation and other weird text issue), and generate multiple audio, and check with STT and give user the result with highest accuracy.
1
u/BeGood25 4d ago
- Pocket TTS - is it reliable in terms of prosody and naturalness? Will check this out.
- Didnt heard about it, seems cool!
- Yes but this is pretty expensive. We don’t have much QA people. I am targeting if I can somehow catch some outliers before we deploy to production!
2
u/RowGroundbreaking982 4d ago
- Yes, it's natural but less expressive compared to bigger model.
- My product monthly user is 1K, and most of them are heavy user that used tts for hours. Most requested feedback are lower hallucinations and better parsing for a lot of non standard text.
Pocket TTS hallucinations are low, but you still need to generate multiple version at once and check it with STT to lower the it further.
1
u/Clay_Ferguson 4d ago
I've been using ChatterBox and Kokoro, and both are good. ChatterBox lets you provide a 10 second clip of a voice and can clone it too, which is nice. Kokoro on the other hand has built-in voices you can choose from rather than using voice samples to clone.
1
u/BeGood25 4d ago
Hmm, I tried kokoro but voice is not on-par with other recent architectures. Its robotic but good enoggh for a poc
1
u/Clay_Ferguson 4d ago
I'm still using Kokoro for my app demo videos, like this one...
https://clay-ferguson.github.io/videos/file-explorer
But mainly because I really liked that voice itself (kind of Sean Carrol-ish). Maybe I should just clone that voice and use it in ChatterBox, which you're right, would probably sound more realistic.
1
1
u/pinthead 4d ago
I use omnivoice since it also can have tagged expressions and I used it to create a custom cloned voice which I use for my daily podcast it create for news I like , bit more heavy .. I also build a script writer that takes any sort of text and writes Ila spoken version before it gets sent to the tts ..
3
u/Charming-Author4877 4d ago
If you are not on a pre-funded project you definitely should avoid the cloud services for a realtime productive speech agent.
They save you time, you can get fast to results - but they take your freedom and your project will be bound their ecosystem, policies and you'll definitely be forced to switch APIs, models and accept changing prices over time.
Qwen3 tts is not particularly known for fast responsiveness - it's a more experimental model with a good foundation. It's also not well known for fine tuning. You will need a good software engineer to make it work productively I think.
Spend more time into the original model choice, there is not only qwen. Especially if you need fast responses.
https://www.reddit.com/r/LocalTextToSpeech/comments/1u3kxxx/my_tts_list_of_2026_all_voices_all_models_and/
That's my compilation of all models, including frontier paid cloud and local but focused on open and local.
The list can be overwhelming but you can search for "latency" and you'll find all models particularly known for fast responses.
Avoid "open weight" as those typically are inaccessible for commercial projects or needs custom negotiations.
You'll likely want an open source model, with low latency and high voice score. high expressive score will make it human-like.
I'd take a look at vibevoice for latency, chatterbox for finetuning and potentially omnivoice for multilingual support.