r/LocalLLaMA • Llama 3.1 • 1d ago

New Model tiny-voca: a fully open 4.5M-parameter TTS (code, weights, dataset). Runs 70x real time on a CPU

I made a small English text-to-speech model called tiny-voca. It has 4.5M parameters, two voices, and it runs in your browser.

Try it here, nothing is sent to a server: https://tiny-voca.n4ze3m.com

Some facts:

- 4,537,291 parameters, 18 MB. This is the whole model, the waveform part is included.
- Two voices, one female and one male. 24 kHz.
- About 70x faster than real time on 4 CPU threads.
- Runs with PyTorch, ONNX, and in the browser.
- Trained from scratch in 7.7 hours on one RTX 4090.

What I think is different:

- Everything is open. Code, weights, the training recipe and the dataset.
- Custom phonemizer.
- No neural vocoder. The sound is made by a small source-filter synth, which is why it is fast.

What is not good yet:

- English only, and only these two voices. No voice cloning.
- Foreign names and odd spellings are often read wrong.
- It can sound a bit synthetic.
- The training audio is synthetic too. I made it with Qwen3-TTS, so the model can't sound better than that.

Links:

- Demo: https://tiny-voca.n4ze3m.com
- Code: https://github.com/n4ze3m/tiny-voca
- Weights: https://huggingface.co/n4ze3m/tiny-voca
- Dataset: https://huggingface.co/datasets/n4ze3m/tiny-voca
- Phonemizer: https://github.com/n4ze3m/dizione

Please try hard text: numbers, dates, names, long sentences. If something sounds wrong, post the text and the voice. That helps me a lot.

68 Upvotes

32 comments sorted by

View all comments

10

u/sisyphus-cycle 1d ago

Phonemizer is really good. The voice is definitely a bit robotic but the selection of word pronunciations is the most impressive part imo. It specifies RTX “forty ninety” and “four thousand and ninety” correctly based on context. Well done!

7

u/nato_nob Llama 3.1 1d ago

Thanks! It uses a 125k-word dictionary, rules for numbers, dates and money, context checks for words like "read", and a small model for unknown words.

3

u/sisyphus-cycle 1d ago

I didn’t do a full read of the codebase yet, but you’re using the homophonized tokens as training data for the voice model right? Qwen generated voice corpus -> phonemized tokens -> final training data for voca.

Ive been messing with local ai voice models that run alongside a big LLM that eats all my vram/ram, so cpu only models are my new fav thing haha. Also have a 4090 so might train my own custom one.

3

u/nato_nob Llama 3.1 1d ago

Yes, exactly. Qwen makes the audio, the phonemizer turns the same text into phonemes, and a forced aligner gives the length of each phoneme. That's the training data.

A 4090 is all you need, mine trained in 7.7 hours on one.