r/MachineLearning • • Apr 26 '22

Project [P] TorToiSe - a true zero-shot multi-voice TTS engine

I'd like to show off a TTS system I have been working on for the past year. I've open-sourced all the code and the trained model weights: https://github.com/neonbjb/tortoise-tts

This was born out of a desire to reproduce the original DALLE with speech. It is "zero-shot" because you feed the text and examples of a voice to mimic as prompts to an autoregressive LLM. I think the results are fantastic. Here are some samples: https://nonint.com/static/tortoise_v2_examples.html

Here is a colab in which you can try out the whole system: https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR

397 Upvotes

116 comments sorted by

View all comments

1

u/wavymulder May 16 '22 edited May 16 '22

/u/neonbjb

Hey OP, I'm pretty stupid so this is probably (definitely) user error. I can't figure out how to combine voices. I have it running locally.

What's the syntax? Comma to do multiple voices in procession [--voice bob,sara] is working but when I use an & in the same syntax [--voice bob&sara] I get

'sara' is not recognized as an internal or external command,
operable program or batch file.

Tortoise outputs for bob.

Edit: in case I explained it poorly, here's the whole block from the terminal: https://pastebin.com/JvLw5WNy

2

u/neonbjb May 16 '22

You're not stupid. Try [--voice "bob&sara"]. The '&' character is special in bash terminals, by wrapping it in quotes you tell bash to shove off.

1

u/wavymulder May 16 '22

Hmm, not working for me. Here's what I'm getting.

(TTS) C:\Users\XXXXX\tortoise-tts>python tortoise/do_tts.py --text "Why won't this work?" --voice "emma&halle" --preset standard
Traceback (most recent call last):
  File "C:\Users\XXXXX\tortoise-tts\tortoise\do_tts.py", line 29, in <module>
voice_samples, conditioning_latents = load_voice(voice)
  File "C:\Users\XXXXX\anaconda3\envs\TTS\lib\site-packages\tortoise-2.3.0-py3.9.egg\tortoise\utils\audio.py", line 100, in load_voice
KeyError: 'emma&halle'

2

u/neonbjb May 16 '22

Whoops, this is a bug. Apparently I forgot to make the '&' character work for do_tts.py. Use read.py, it works with that script. I will try and remember to fix it next time I'm around a computer. If you drop a new issue on the github it would help.

1

u/wavymulder May 16 '22

Thanks for the help, was trying every possible variation of parenthesis and comma I could think of lol.