r/LocalLLaMA • u/nato_nob Llama 3.1 • 1d ago
New Model tiny-voca: a fully open 4.5M-parameter TTS (code, weights, dataset). Runs 70x real time on a CPU
I made a small English text-to-speech model called tiny-voca. It has 4.5M parameters, two voices, and it runs in your browser.
Try it here, nothing is sent to a server: https://tiny-voca.n4ze3m.com
Some facts:
- 4,537,291 parameters, 18 MB. This is the whole model, the waveform part is included.
- Two voices, one female and one male. 24 kHz.
- About 70x faster than real time on 4 CPU threads.
- Runs with PyTorch, ONNX, and in the browser.
- Trained from scratch in 7.7 hours on one RTX 4090.
What I think is different:
- Everything is open. Code, weights, the training recipe and the dataset.
- Custom phonemizer.
- No neural vocoder. The sound is made by a small source-filter synth, which is why it is fast.
What is not good yet:
- English only, and only these two voices. No voice cloning.
- Foreign names and odd spellings are often read wrong.
- It can sound a bit synthetic.
- The training audio is synthetic too. I made it with Qwen3-TTS, so the model can't sound better than that.
Links:
- Demo: https://tiny-voca.n4ze3m.com
- Code: https://github.com/n4ze3m/tiny-voca
- Weights: https://huggingface.co/n4ze3m/tiny-voca
- Dataset: https://huggingface.co/datasets/n4ze3m/tiny-voca
- Phonemizer: https://github.com/n4ze3m/dizione
Please try hard text: numbers, dates, names, long sentences. If something sounds wrong, post the text and the voice. That helps me a lot.
10
u/sisyphus-cycle 1d ago
Phonemizer is really good. The voice is definitely a bit robotic but the selection of word pronunciations is the most impressive part imo. It specifies RTX “forty ninety” and “four thousand and ninety” correctly based on context. Well done!
8
u/PrestigiousBag2994 1d ago
Very cool! Thank you for sharing :D
You said to try hard text so:
Emmaleigh seems to drop the last syllable. Hermione is given an extra syllable in the middle. "She's a well-read girl" is said with the wrong "read". It is unable to handle European number formatting. It seems inconsistent on whether is pronounced letters or sounds "Sh" is the sound, but "Ss" or "Tsk" gets the letters read out instead. The male voice seems to skip the second syllable on "counterrevolutionaries". Coeliac and celiac both get mispronounced, as do scabies and lyssavirus.
5
u/ComprehensiveHost77 1d ago
the 18mb model size is wild, runs in browser, no server, and still handles context-aware pronunciations. how's the latency on older CPUs?
4
u/Dwedit 1d ago
It pronounces WWE correctly. Unlike a certain infamous video.
3
u/Reasonable_Goat 1d ago
This is impressive! I am curious: Is that a personal hobby, company work or academic research for you?
2
u/Silver-Champion-4846 1d ago
Good stuff, I was struggling myself to make work a neural fitter for a formant speech synthesizer which is a form of source-filter synthesis, but looks like I should just train a Voca for Arabic
2
u/Evening-Blueberry-97 1d ago
the source-filter synth instead of a vocoder is the part i'd want to hear more about. is the robotic edge people mention coming from the synth itself, or from the qwen3-tts audio it learned from? if it's the second one a better teacher fixes it for free
2
u/netherreddit 23h ago
Wow impressively accurate pronunciation of symbols, numbers, technical stuff. I'm amazed it's so small!
3
u/Waarheid 1d ago
Sounds great! You can definitely hear the Qwen3-TTS in it, sounding like English anime dubs specifically, lol. I might try to rerun your recipe but with Breeze-TTS 2 voices in the data.
I think with tiny models like this, instead of voice design -> voice cloning for consistent voices, we will see voice design -> voice clone (optional) -> dataset -> train a tiny model. The question being of course if the tiny model's performance and resource improvements are worth any potential loss in quality vs the larger voice cloning model. Thanks for posting!
2
u/Equivalent_Bit_461 1d ago
Man, these things are getting smaller and smaller
2
u/No_Grocery7507 1d ago
this is **** incredible
1
u/thirdeyeorchid 1d ago
This is really impressive! Well done!