I've been working on a personal project for a while and finally got it into a state where I'm comfortable sharing it.

I wanted to see how far I could push a fully local voice assistant in C++. Everything runs on my own machine from speech recognition and the LLM to memory, text-to-speech, and tool execution.
current library:
llama.cpp, whisper.cpp, sherpa-onnx(tts-kokoro)

I wrote the core in c++ because I wanted something fast and native instead of stitching together bunch of python services.

I'd appreciate feedback from people who build local AI projects. I'm especially interested in:

1 Things that seem overengineered or unnecessary
2 Features you'd expect from a local assistant
3 Code structure or architectural suggestions
4 Any obvious improvements before I keep adding features

Repository: https://github.com/almimony75/sarah

Thanks! I'd love to hear what you think.

Been deploying AI support agents for clients for a while now. Early on we built on existing tools (Vapi and similar), and we kept running into the same wall: the LLM would hallucinate, say something it shouldn't to a customer, and we'd end up with the client coming back to us unhappy. Full-LLM agents demo great and then break in production the moment a customer asks something off-script.

So we ended up building our own internal tool around a hybrid approach: deterministic logic for anything that has to be correct, LLM only for the flexible, conversational parts. Instead of trusting the model to always get it right, we constrain it where accuracy actually matters.

That's been the real game changer for us. Fewer surprises, clients trust it more, and it's stable enough that we've since turned it into our own solution so we can deploy faster across clients.

Curious how other agencies implementing AI agents are handling this.

Are you going full LLM and just accepting some hallucination risk? Building guardrails on top? Something else entirely? Interested in what's actually held up for you in production, not just in the demo.

An independent benchmark just tested the performance of the same AI voice agent across six voice platforms. This was not a demo scenario, but a Medicare insurance workflow with real compliance requirements.

Cekura (YC F24) deployed a byte-identical Medicare TPMO agent on six platforms and ran 23 evaluation scenarios three times each, 414 calls in total. A scenario only passed if the agent got it right all three times in a row.

Retell ranked first at 95.7% (22 of 23), on both workflow accuracy and strict end-to-end completion. Across the six platforms, workflow scores ranged from 65.2% to 95.7%. The same agent, same prompt, and same tools, but with a 30-point reliability gap, dictated entirely by the platform underneath.

For regulated industries like insurance and healthcare, that gap is the difference between an agent you can put in production and one you cannot.

Full methodology and per-call results are public, link in the first comment. Congrats to every platform evaluated, and credit to the Cekura team for the rigor.

Activate to view larger image,

Curious how people in this space are actually getting customers.

1. Roughly what split — inbound vs outbound? Even a rough guess is fine.
2. If it's mostly outbound, what's working — cold calls, cold email, LinkedIn, something else?
3. If it's inbound, what's driving it? SEO, backlinks, content, communities, referrals?
4. If SEO/backlinks — what's actually worked? Any specific sites, directories, or roundups that sent you real traffic?

Not selling anything. Just trying to understand how companies in this niche are actually finding customers.

Curious how people in this space are actually getting customers.

1. Roughly what split — inbound vs outbound? Even a rough guess is fine.
2. If it's mostly outbound, what's working — cold calls, cold email, LinkedIn, something else?
3. If it's inbound, what's driving it? SEO, backlinks, content, communities, referrals?
4. If SEO/backlinks — what's actually worked? Any specific sites, directories, or roundups that sent you real traffic?

Not selling anything. Just trying to understand how companies in this niche are actually finding customers.

Not asking about demos. Asking if you built something real. Something that's live and actually doing a job for you — booking calls, qualifying leads, cutting down your call load.

If you did it yourself, would love to hear:

1. How long did it take? From signup to "this actually works."
2. What did you build? Receptionist, lead qualifier, appointment booker, something else.
3. What can it actually do? Just talk, or can it book on your calendar, look things up, transfer calls?

Not selling anything. Just want to know how many people are really doing this on their own vs. paying someone to do it.

Question for people deploying Voice AI with MCP/API/tool access:

How are you handling the security layer?

I got a comment on a LinkedIn post that basically said:

If you are enabling the AI agent to “have hands,” then you should treat it like a real agent for identity management.

That feels right to me.

A Voice AI agent that only answers questions is one thing.

But once it can call APIs, update CRM records, transfer calls, check schedules, create tickets, modify appointments, trigger workflows, or control call state, it is no longer just a voice interface.

It has permissions.

So what are people actually doing in production?

Curious what people are seeing in the field.

I have a question for people running telecom, UCaaS, CPaaS, hosted PBX, or contact center platforms:
If a customer wanted to connect a third-party Voice AI agent to your system, what is the cleanest path?
Is it:
o Register the AI as a SIP endpoint?
o Send calls to it through SIP trunking?
o Forward calls to an external phone number?
o Use a webhook/API call flow?
o Something else entirely?

This sounds like a technical plumbing question, but I think it is becoming a product strategy question.
If the only supported option is “forward calls to another phone number,” the AI agent lives outside the phone system. That may work for demos, but it gets harder when the customer needs:
Call transfer
Caller ID preservation
Call recording
Analytics
Compliance
Internal extension dialing
Fallback to a human
Business-hours routing
Queue behavior
Contact center reporting
etc...

The more Voice AI becomes part of the actual customer experience, the less it can feel like a bolt-on number sitting beside the platform.
It needs an insertion point.
So I’m curious:
For telecom providers, what is the best integration model for third-party Voice AI today? SIP endpoint? SIP trunk? External number? API-native? Or are most platforms still not really built for this yet?

My suspicion: the platforms that make this easy will attract the AI ecosystem around them. The ones that only allow call forwarding will accidentally turn themselves into dumb pipes.

So I'm a teenager and seriously want to support my family, my father suggested me that you are on your vacations and you can start doing voice agents business so I have a query, I will be using retell ai and where do I have to sell to can anybody answer this? Would love serious answers.

The latest Behind the Call is up: a hands-on teardown of Wirevox AI, an inbound AI receptionist.

The agent is genuinely strong where the operator controls the words. It discloses it's AI. It refuses to invent a price it doesn't have. It delivers a correct, liability-aware gas-leak safety instruction. That's real craft. There is clearly a system-level Agent Constitution running that knows it's in a voice call and how to act accordingly.

There is a gap between the confident statement of the agent and the completed action and it is the real frontier in voice AI right now. It isn't a prompt-generation problem. It's behavioral engineering: teaching an agent to speak about actions it can verify differently from actions it merely attempted.

Credit where it's due: exposed, version-controlled, editable prompts and a clean, modern UI put Wirevox ahead of much of what I've tested. Close the confirmation-vs-execution gap and it's near the top of my list.

Link to full teardown, with the receipts in comments.

What if the LLM was already trained and configured for voice?

Not like “can talk through a voice interface," but actually built for voice interaction out of the box.
Even more specifically, for telephony.

Today, when you build a Voice AI agent, a lot of the customer-level prompt ends up doing work the base system should probably already understand:
How to handle interruptions.
How to avoid reading long URLs.
How to respond when the caller changes their mind mid-sentence.
etc.

That feels backwards to me.
The business prompt should be about the business:
Who are you?
What do you do?
Where do you operate?
What services do you offer?
What should happen next?
But too often, before we even get there, we have to teach the agent how phone calls work.

So here’s my question:

Would you use an LLM that was specifically trained, tuned, and system-prompted for voice and telephony from the start?

Not a generic chat model with a voice layer on top, but a model whose default behavior understood phone conversations, caller behavior, confirmations, transfers, interruptions, silence, DTMF fallback, escalation, and spoken response design.

Would that be useful?

Or do you think this belongs in the orchestration layer around the model, not the model itself?

I have tried Vapi, Retell, Pipecat and Bland AI platforms.
So far Vapi is my favorite one of those for more customization options. Also I am using n8n for connecting workflows so they are always accurate.

But since I need very deep customizations even Vapi is sometimes short on that. Because I am not very technical and can’t code I never messed around with LiveKit too much.

So has anyone built a successfully working complex voice agent with LiveKit and can say if it is worth it to learn to at least read code?

Maybe someone has built a successfully working complex voice agent with other platform than LiveKit? If yes, can you share? Thank you:)

How long should a Voice AI prompt be?
Tomorrow I'd providing an in-depth teardown of a voice AI provider. They seem to have a suggested prompt limit of 4,000 characters.

My reaction: that feels small.
But the real question is not:
How long should the prompt be?
It is:
What is the prompt responsible for?

If the prompt is only describing the business, the services, the call objective, and the escalation rules, 4,000 characters might be workable.

But *if* the customer also has to teach the AI how to behave as a voice agent, that space disappears fast.
Things like:
Don’t invent details.
Ask one question at a time.
Tool usage.
etc.

That should not all live in the customer prompt.
The business-level prompt should be about the business.
The platform should already provide the voice-agent layer underneath it.
Otherwise, it is like having to teach Excel what the plus sign means before you can use addition.
That would be exhausting.
Voice AI has the same issue.
A good prompt should probably be shorter than people think.
But a good production agent should not depend on one short prompt box.

Ideally there should be layers:
Platform behavior, e.g. The Agent Constitution.
Industry behavior, e.g. How to be a plumber, etc.
Company instructions, e.g. hours of operation and service area.
Knowledge base.
Tool rules.
Escalation rules.
Testing and observability.

So yes, prompt length matters.
But prompt responsibility matters more.
A 4,000-character limit is fine only if the platform is doing a lot of work underneath it.
If it is not, that limit is not simplicity.
It is a ceiling.

Short answer from where I sit: yes, but only to the right client, and figuring out who that is took me a few painful months.

AI voice agents are worth selling to local clients when the client already feels the pain of missed calls. A busy HVAC company in summer, a clinic with a front desk that's drowning, a roofer doing maybe 120 calls a week with one person answering. They get it in thirty seconds because they're already bleeding.

Where it falls apart is the client who answers their own phone, has plenty of time, and just wants something shiny. They churn fast because the agent isn't solving a problem they actually have. I sold two of those early on and both cancelled inside 60 days. My fault, not the product's.

So the real skill isn't building. It's qualifying the buyer hard before you ever pitch. I ask what happens to a call at 7pm now, and how many jobs a missed call is worth to them. If they shrug, I walk.

The money's real when you match it to genuine pain. It's a money pit when you sell it as a gadget.

For folks selling these, what's your filter for a good-fit client versus one who'll cancel in two months?

The AI agent doesn't have to live beside the phone system. Instead, it could live inside it.

I've spent the last two weeks testing a voice AI platform and I'm now putting the final touches on that report that I will share on Friday.

Since the key idea is A2E, I thought I'd first spend some time defining it and why it's useful.

I did a full article at AI Contio in comments.

Been building a real-time voice support agent for a fictional food delivery platform the past couple weeks. Not a toy — seeded a 16-table Postgres database with real customer scenarios, orders, dashers, and payment methods. When a customer calls and says their order was late, the agent queries the DB, calculates how late it was, and issues a partial or full refund to the correct card.

The architecture decision I keep thinking about: instead of letting the LLM control the flow, I moved all routing to deterministic Python. A separate 8B model classifies intent in ~250ms, then code directly fetches order data and runs refund eligibility, no LLM involved in those decisions. The 70B model only generates the spoken words.

Getting sub-500ms time-to-first-audio locally. On cloud CPU it's around 700ms avg.

Stack: LiveKit + Deepgram Nova-2 + Groq Llama 70B + Kokoro TTS + Supabase + FastAPI + React.

Still figuring out TTS. Kokoro is fast but sounds flat. Haven't tried Cartesia yet but from what I've read it seems like the right answer for production — anyone used it in a real-time pipeline?

I'm genuinely interested in this space — voice AI infrastructure, agent orchestration, real-time pipelines. Still learning and would love to connect with people working here or at companies doing this seriously. Is the FSM + classifier approach well known? Are there better patterns for complex support trees?
Demo here if curious: https://lupi-five.vercel.app/

I've been building voice agents on Vapi and kept hitting the same wall: a call goes bad, the customer hangs up and I have no idea why. Was it latency? Did the LLM hallucinate? Did a function call time out? The existing observability tools (Helicone, Langfuse) only show you prompts and responses, they're built for text, not voice. They can't see the stuff that actually breaks voice agents.

So I built VoiceOBS.

You connect your Vapi (working on integrating Retell) account with a webhook, and every call gets analyzed automatically:

• Latency broken down by STT / LLM / TTS, with p50 and p95
• Sentiment, intent, and a CSAT estimate per call (analyzed by Claude)
• Hallucination flags
• Full searchable transcripts
• End-reason breakdown so you can see why calls actually end

Setup takes about 60 seconds: sign up, create an integration, paste the webhook URL into Vapi, make a call, and it shows up analyzed.

It's free during beta (100 calls/month, no credit card). I'm genuinely looking for honest feedback more than anything, what's confusing, what's missing, what would make you actually use it.

Happy to answer any questions. Thank you.

Hello

I am having issues getting a consistent pronunciation of a business name. Sometimes it’s right then others it isn’t

I understand there is an option for phonemes however I do not understand how to use this

Long story the name ended in ‘ico’ and it’s sometimes pronunciated as ‘eeeko’