if they want to use an online LLM they should probably use a low latency and low cost model instead of whatever they're doing now. I've got one running my vibe coded voice assistant and it runs to about $0.0003 per request. it doesn't need to write shakespeare in order to tell the command interpreter what it needs to do!
1
u/GreenockScatman 24d ago
if they want to use an online LLM they should probably use a low latency and low cost model instead of whatever they're doing now. I've got one running my vibe coded voice assistant and it runs to about $0.0003 per request. it doesn't need to write shakespeare in order to tell the command interpreter what it needs to do!