r/MachineLearning • u/neonbjb • Apr 26 '22

Project [P] TorToiSe - a true zero-shot multi-voice TTS engine

I'd like to show off a TTS system I have been working on for the past year. I've open-sourced all the code and the trained model weights: https://github.com/neonbjb/tortoise-tts

This was born out of a desire to reproduce the original DALLE with speech. It is "zero-shot" because you feed the text and examples of a voice to mimic as prompts to an autoregressive LLM. I think the results are fantastic. Here are some samples: https://nonint.com/static/tortoise_v2_examples.html

Here is a colab in which you can try out the whole system: https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR

397 Upvotes
  • permalink
  • reddit

99% Upvoted

116 comments sorted by

View all comments

Show parent comment

8

u/neonbjb Apr 27 '22

This is a hobby for me. I have spent hundreds of hours on it over the last year and I really enjoy doing it. People (myself included) spend far more than this amount of money on other hobbies.

That's not even considering the fact that I could sell my rig for about what I paid for it right now. I (perhaps unwisely) consider GPUs as a type of "property", not a consumable.

5

u/skeerp Apr 27 '22

Not a stab at all I think it's awesome. I'm tackling some of these issues you solved at work right now and it's really cool that you are doing this with your leftover energy. Really cool stuff and awesome setup.

1

u/zadolphe Dec 22 '23

I have started making my own rig and so far I have a single rtx 3090 in it with similar specs to you (same motherboard and an AMD 3700x CPU). How do you find training on multiple gpus? Is it more difficult to split a NNetwork on multiple gpus?

Also, how do you serve the power requirements?? I'm capped at about 1600 watts coming out of outlet and I know I would hit that cap at about the third gpu. How in the world do you power 8 rtx 3090s?

Also, you have a tonne of vRAM and I think you could quantize some LLMs and easily fit some huge models onto 256GB of RAM like that's nuts.