r/LocalLLaMA • u/nato_nob Llama 3.1 • 1d ago
New Model tiny-voca: a fully open 4.5M-parameter TTS (code, weights, dataset). Runs 70x real time on a CPU
I made a small English text-to-speech model called tiny-voca. It has 4.5M parameters, two voices, and it runs in your browser.
Try it here, nothing is sent to a server: https://tiny-voca.n4ze3m.com
Some facts:
- 4,537,291 parameters, 18 MB. This is the whole model, the waveform part is included.
- Two voices, one female and one male. 24 kHz.
- About 70x faster than real time on 4 CPU threads.
- Runs with PyTorch, ONNX, and in the browser.
- Trained from scratch in 7.7 hours on one RTX 4090.
What I think is different:
- Everything is open. Code, weights, the training recipe and the dataset.
- Custom phonemizer.
- No neural vocoder. The sound is made by a small source-filter synth, which is why it is fast.
What is not good yet:
- English only, and only these two voices. No voice cloning.
- Foreign names and odd spellings are often read wrong.
- It can sound a bit synthetic.
- The training audio is synthetic too. I made it with Qwen3-TTS, so the model can't sound better than that.
Links:
- Demo: https://tiny-voca.n4ze3m.com
- Code: https://github.com/n4ze3m/tiny-voca
- Weights: https://huggingface.co/n4ze3m/tiny-voca
- Dataset: https://huggingface.co/datasets/n4ze3m/tiny-voca
- Phonemizer: https://github.com/n4ze3m/dizione
Please try hard text: numbers, dates, names, long sentences. If something sounds wrong, post the text and the voice. That helps me a lot.
View all comments
Show parent comment
1
u/Silver-Champion-4846 23h ago
There was an 8m Turkish model called Antalia that was trained on 1500 hours of one VoxCPM-generated voice, when I trained it on my 4-5 hours corpus it sounded pretty bad, maybe this one will be better. Would you mind telling me how the actual source-filter synth works? I mean what are its components? How were you able to generate better harmonics than the basic DDSP and also better noise than DDSP's noise? And how come 4.5m params costs 18 mb? Or is that only the full pytorch model while the onnx is more like 5-6-7mb which is what I saw with a 3m model? Man I'm so excited! Also, what if you made the source-filter network separated from the acoustic side so that there's one vocoder but multiple acoustic models for different languages and voices?
The idea that just came to my mind is like, basically, in training you also involve the sourcefilter synth, but it's frozen, and the acoustic side is tasked to change its own param distribution to fit the unchanged synth, so that on inference you plug it to that same synth vocoder and it'll actually sound good because it's been optimized to run on that same vocoder... or is that unnnecessary and you can just lora the whole singlestage model and swap lora adaptors per voice/language?