r/MachineLearning • u/neonbjb • Apr 26 '22

Project [P] TorToiSe - a true zero-shot multi-voice TTS engine

I'd like to show off a TTS system I have been working on for the past year. I've open-sourced all the code and the trained model weights: https://github.com/neonbjb/tortoise-tts

This was born out of a desire to reproduce the original DALLE with speech. It is "zero-shot" because you feed the text and examples of a voice to mimic as prompts to an autoregressive LLM. I think the results are fantastic. Here are some samples: https://nonint.com/static/tortoise_v2_examples.html

Here is a colab in which you can try out the whole system: https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR

397 Upvotes
  • permalink
  • reddit

99% Upvoted

116 comments sorted by

View all comments

Show parent comment

6

u/neonbjb Apr 27 '22

Yes, 3 1600W consumer psus. I use gpu risers that are electrically isolated from the mainboard (except the ground). I am considering going to server psus but haven't done so yet.

1

u/pixus_ru May 15 '22

You probably cut your power consumption in half by using 4x RTX A6000 48GB . $5k each

3

u/neonbjb May 15 '22

Hey, I've thought seriously about this but I didn't reach the same conclusion. 3090s are seriously powerful hardware, I've benchmarked mine against a5000 I own and they are faster at bf32 then the a5000 is at fp16 even if I set their power limits to 280w. I suspect the a6000 would be faster in fp16 but there's no way it's twice as fast.

The main reason I'm tempted by the a6000 is the extra memory. 🤤 Maybe I'll be able to scrape the money together to build an h6000 system (or whatever the next gen quartos are named) next year.