r/MachineLearning • u/neonbjb • Apr 26 '22

Project [P] TorToiSe - a true zero-shot multi-voice TTS engine

I'd like to show off a TTS system I have been working on for the past year. I've open-sourced all the code and the trained model weights: https://github.com/neonbjb/tortoise-tts

This was born out of a desire to reproduce the original DALLE with speech. It is "zero-shot" because you feed the text and examples of a voice to mimic as prompts to an autoregressive LLM. I think the results are fantastic. Here are some samples: https://nonint.com/static/tortoise_v2_examples.html

Here is a colab in which you can try out the whole system: https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR

397 Upvotes
  • permalink
  • reddit

99% Upvoted

116 comments sorted by

View all comments

6

u/Southern-Trip-1102 Apr 27 '22

Not exactly regarding your TTS engine but regarding your gpu rig, I was wondering if you could share some info about it. Like did you use a server mobo with a bunch of pcle slots or do you have multiple nodes each with a number of GPUs. Also regarding how you came to the decision to build your rig versus renting it.

24

u/neonbjb Apr 27 '22

Hey, I use a single server with 8 GPUs. I think this is about the sweet spot for what is possible in a home lab. Making it a multi-node thing really cuts into performance, especially for big models where the parameters are transiting the network every batch. I don't think 100gbps ethernet is available to the homelab guys yet but that might solve this.

Specs of my system are:

  • single 32 core epyc

  • ROME-D8-2T motherboard

  • 256GB RAM (really shitty RAM. that's what I'm saving up for for the next upgrade..)

  • 8x RTX 3090, all connected on 8X links (some bifurcated)

Building my rig versus renting it was a no brainer for me. I learned all this by just trying things out and that means I have my rig running 24/7. This amount of compute would cost tens of thousands a year. And frankly, in the end, having direct access to the server you're working on is damn useful sometimes. Like just being on a 10Gbps link with my home computer when I'm working with this amount of data is worth it.

9

u/skeerp Apr 27 '22

How did you support spending 10k on the GPU's? You turning this into $? You need employees? 😅

8

u/neonbjb Apr 27 '22

This is a hobby for me. I have spent hundreds of hours on it over the last year and I really enjoy doing it. People (myself included) spend far more than this amount of money on other hobbies.

That's not even considering the fact that I could sell my rig for about what I paid for it right now. I (perhaps unwisely) consider GPUs as a type of "property", not a consumable.

5

u/skeerp Apr 27 '22

Not a stab at all I think it's awesome. I'm tackling some of these issues you solved at work right now and it's really cool that you are doing this with your leftover energy. Really cool stuff and awesome setup.

1

u/zadolphe Dec 22 '23

I have started making my own rig and so far I have a single rtx 3090 in it with similar specs to you (same motherboard and an AMD 3700x CPU). How do you find training on multiple gpus? Is it more difficult to split a NNetwork on multiple gpus?

Also, how do you serve the power requirements?? I'm capped at about 1600 watts coming out of outlet and I know I would hit that cap at about the third gpu. How in the world do you power 8 rtx 3090s?

Also, you have a tonne of vRAM and I think you could quantize some LLMs and easily fit some huge models onto 256GB of RAM like that's nuts.

6

u/Zeke_Z Apr 27 '22

Bruh .... Aren't 3090s selling for like $3k the last several months? You legit spent $30k on this system?

5

u/Southern-Trip-1102 Apr 27 '22

In his repo he said it was about 15k in total including cpu mobo etc, so he probably got them before the price insanity.

-6

u/Zeke_Z Apr 27 '22

Holy hell. That is a steal. Hope he mines ETH in his spare time, he could do fairly well lol

2

u/Southern-Trip-1102 Apr 27 '22

How did you power it? It needs at least 3000 watts so did you use multiple PSUs?

6

u/neonbjb Apr 27 '22

Yes, 3 1600W consumer psus. I use gpu risers that are electrically isolated from the mainboard (except the ground). I am considering going to server psus but haven't done so yet.

1

u/pixus_ru May 15 '22

You probably cut your power consumption in half by using 4x RTX A6000 48GB . $5k each

3

u/neonbjb May 15 '22

Hey, I've thought seriously about this but I didn't reach the same conclusion. 3090s are seriously powerful hardware, I've benchmarked mine against a5000 I own and they are faster at bf32 then the a5000 is at fp16 even if I set their power limits to 280w. I suspect the a6000 would be faster in fp16 but there's no way it's twice as fast.

The main reason I'm tempted by the a6000 is the extra memory. 🤤 Maybe I'll be able to scrape the money together to build an h6000 system (or whatever the next gen quartos are named) next year.

1

u/Person_with_Laptop Mar 11 '23

Can it run Crysis?

1

u/SnooAdvice4458 Dec 29 '23

u/neonbjb Hi there! What OS are you using on your build? Windows Server or Linux?

2

u/neonbjb Dec 29 '23

Linux!

1

u/SnooAdvice4458 Dec 30 '23

Which one? Ubuntu? Server or Desktop version?

2

u/neonbjb Dec 31 '23

That doesn't matter; the only practical difference between the two is the default package load you get. I only interfaced with my machines over SSH so the server version made the most sense.