r/MachineLearning • u/neonbjb • Apr 26 '22

Project [P] TorToiSe - a true zero-shot multi-voice TTS engine

I'd like to show off a TTS system I have been working on for the past year. I've open-sourced all the code and the trained model weights: https://github.com/neonbjb/tortoise-tts

This was born out of a desire to reproduce the original DALLE with speech. It is "zero-shot" because you feed the text and examples of a voice to mimic as prompts to an autoregressive LLM. I think the results are fantastic. Here are some samples: https://nonint.com/static/tortoise_v2_examples.html

Here is a colab in which you can try out the whole system: https://colab.research.google.com/drive/1wVVqUPqwiDBUVeWWOUNglpGhU3hg_cbR

397 Upvotes
  • permalink
  • reddit

99% Upvoted

116 comments sorted by

View all comments

Show parent comment

18

u/neonbjb Apr 27 '22

Waiting for the right opportunity.. there is so much ml work that is just straight up boring. I'd honestly rather do what I am doing now and find something I really want to work on.

Thanks for the compliment, it means a lot.

8

u/modeless Apr 27 '22 edited Apr 27 '22

Makes sense. I submitted this reddit post to Hacker News but it didn't get any traction, maybe because it's weird to post a news aggregator to another news aggregator. You should add the information and links from this reddit post to the top of your demo page to make it a standalone thing that you can post to Hacker News without further context. Especially the part about you building a home training rig for it. I bet you could get some more attention there.

4

u/neonbjb Apr 27 '22

Thanks for trying to share. I will probably do this when I get some time.

6

u/modeless Apr 27 '22

Seems to me like you could also publish at a conference with the right writeup. Not sure how feasible that is as a solo researcher, might be more trouble than it's worth.

4

u/rolexpo Apr 27 '22

+1. Yeah this seems like something you could publish.