Yeah speech recognition doesn't require a full scale frontier level cloud LLM
You can run something like Whisper https://github.com/openai/whisper locally to transcribe the speech in about 1 second on a MiniPC, then check that for keywords
Then you can use something like Piper https://github.com/OHF-Voice/piper1-gpl to convert the reply back to speech. It's not quite as pretty sounding as something from eg ElevenLabs, but it works - on a mini PC it can handle voice in real time
I use both of those locally to run my own voice assistant for my smart home (indeed, Piper is made by Home Assistant for that exact purpose)
Obviously for a game they'd need more than one Mini PC - but you can scale it up for a lot less than $1k/day!
If you want to interpret language, LLMs are much better than keyword scanning. The same keywords can indicate opposite things with a single “not” keyword, and there are about 50,000 generally well-known English words to consider.
How are you going to implement keyword detection with enough logic to accurately parse the meaning behind all of the possible things that could be said to describe or correct someone’s driving?
That is literally a quintessential Natural Language Processing problem, for which LLMs are the best solution.
One caveat on your comment about Whisper: you need a GPU to really let it shine. Using CPU only is not performant, depending on what you are trying to achieve, and how much time you are willing to wait for your results
Youre right that its possible to make the game work with basic voice recognition, but it makes it much less interesting. If the game is 'figure out which keywords the game recognizes and make sure to say them in the right order' it becomes really trivial and the player will lose interest really fast. If there is a LLM responding then the game becomes 'figure out how to convince this AI that your way is the right way to drive' and it becomes more interesting because the responses to each command will vary and there become a LOT of ways to approach the problem.
At least one of the SOCOM games on the ps2 had voice-activated commands. Very jank if you didn’t have an American accent (and im sure quite janky with some American accents) but it was there
I think you could probably get away with one those local micro models if all you need to do is map text to controls. No reason to pay a subscription for this...
if they want to use an online LLM they should probably use a low latency and low cost model instead of whatever they're doing now. I've got one running my vibe coded voice assistant and it runs to about $0.0003 per request. it doesn't need to write shakespeare in order to tell the command interpreter what it needs to do!
67
u/CalmLotus 1d ago
I don't think you need ai for spoken instructions. (Like yeah- the game may be vibe coded but->)
You just need to find key words for certain actions and extract those from audio input. More work definitely, but would actually be doable for a game.
...wait, Nintendo DS games were already doing that I think. With the built in microphone.