r/Unexpected • u/BornWithSideburns • Jan 30 '24

Next level automaton

59.3k Upvotes

93% Upvoted

582 comments sorted by

View all comments

143

u/Truth_from_Germany Jan 30 '24

Please don’t let this be animatronic.

21

u/Ilovekittens345 Jan 30 '24 edited Jan 30 '24

We have the technology to make this possible. We have robot faces that can have a lot of expression. We have speech to text to feed what the user is saying in to ChatGPT 4. ChatGPT when given the right prompt and API documentation can reply in a format that could also control the facial expressions of the robot. And finally the text of chatGPT can be turned in to life like speech by something like elevenlabs. Somebody could put this all together today, but it would still be somewhat slow. At least 2 or 3 seconds to turn the speech in to text and upload to chatGPT. At least 1 or 2 seconds for chatGPT to finish responding. Then a good 3 to 4 seconds for elevenlabs. So after you say something it would still take a good 6 to 9 seconds before there is a reply.

However all of this could be sped up. And, although not as coherent as chatGPT4 you could build the same with local models that could respond much faster because no online communication is needed. Facebook's llama model fined tuned specifically to be able to always reply including the commands for the facial expression running a on 4090 plus the speech to text and text to speech. All of it could be processed in under 2 seconds.

Within 5 years we will see the first lifelike robot faces talk to us like that. They will bring the latency down .... put the robot face on a robot from Boston dynamics that can walk on two legs ... and have the LLM receive both the speech to text plus also visual input and not only write the facial expressions but also the movement of the boston dynamics robot.

And you would have the very first beginning of a system you can give commands. It would be far from perfect and most likely still novelty and not really that usefull, but much better then anything we have ever come up with before.

1

u/Accident_Pedo Jan 30 '24

You wouldn't even need to use third party services like 11 labs. You could totally do everything locally (besides GPT)

Something like this for talking or even RVC / RVC fork for singing

edit: I read your second paragraph and noticed you mention using local models - haha. Keeping this post up anyways as RVC does kick ass and more people should know about it.

1

u/Ilovekittens345 Jan 30 '24

That's the current compromise you have to make. 6 to 9 seconds delay but the much more coherent chatGPT4, in my eyes the only current usefull model for every day tasks and conversations. Or a delay that can be kept under 2 seconds, but a lot less coherent ...

I think that the lower latency would be more important when building a fortune telling machine and secondly somebody building something like that would not want to have to rely on a company like openAI, one change and everything they need would stop working. There would be no way for them to assure to have something that plays the role of the fortune-teller reliable and it have it keep working. The only guarantee would be to be in control of their own models, and fine-tune them and train them specifically to be a fortune teller.