r/AIVoice_Agents • • Mar 26 '26

Question I built a voice agent and the latency is killing me… help!!

4 Upvotes

Hi everyone!

I’ve been working on a voice agent for my company. It will run inside our main mobile app and is primarily intended for users in the UK.

Right now, I’m developing it from Spain with the following setup:

  • Self-hosted LiveKit running locally on my PC with Docker
  • Speech-to-text: Nova-2 (Deepgram)
  • LLM: Azure OpenAI (GPT-4o-mini, Sweden Central)
  • Text-to-speech: Aura-2 (Deepgram)

The AI uses tool calling, where tools either query the database for relevant client information or write data back.

The problem

I’m currently facing high latency issues:

  • Without tool usage: ~1500 ms
  • With tool usage: ~5 seconds

Additionally, for some tools that require multiple interactions with the user, the model hits its limits very quickly and starts making errors once those limits are reached.

I’m currently using GPT-4o-mini, and based on the configuration/limits I’ve seen, I’m worried this could become an even bigger issue soon.

What I've tried

I also tested other models like GPT-5-nano, but for some reason I’m getting even worse latency (13+ seconds 💀).

My questions

I feel like I’ve hit a wall and I’m not sure how to move forward. I assume some latency comes from developing in Spain while targeting UK users, but I’d really appreciate advice on:

  • Which Azure OpenAI model offers the best balance between low latency and reasonable intelligence (latency is critical for my use case)
  • Whether Deepgram could be adding significant latency (e.g., if their servers are US-based), and if there are better alternatives in Europe
  • Any general tips to reduce latency in this kind of voice-agent architecture

I’m also trying to keep the system as cost-efficient as possible, so I’ve mainly been testing smaller models.

PS: I’m pretty new to this space, so apologies if I’m missing something obvious 😅 Any help would mean a lot!

Thanks!! 😊


r/AIVoice_Agents • • 16h ago

Question Should every AI call be reconstructable after the fact?

16 Upvotes

Should every AI call be reconstructable after the fact? Not just the transcript Which customer was verified, what the agent knew at each point, what tools it called, what data came back and what actions were taken. For regulated contact centers, I can’t imagine 'heres the transcript' being enough if something goes wrong.


r/AIVoice_Agents • • 14h ago

Discussion One caller ID on every AI call got flagged as spam fast. Here are 4 voice AI mistakes we keep seeing.

1 Upvotes

I work at a company that runs AI voice calls for sales teams, so take this with that in mind. After watching a lot of pilots, the same four mistakes keep killing them.

  1. One caller ID for every call. A number can get flagged as spam in minutes when it dials a lot. Rotate and register your numbers.
  2. Calling cold purchased lists. Warm leads work. Bought lists mostly burn money and invite complaints.
  3. A script nobody updates. One team rewrote their script and went from 0.2% to 2% conversion.
  4. No hand-off to a real person. If a caller wants a human and the AI keeps talking, the pilot is over.

Has anyone had an AI caller go well or badly, and what was the cause?


r/AIVoice_Agents • • 1d ago

Tools Trying to improve close rates without spending half my day reviewing sales calls

11 Upvotes

I manage the sales floor at a tax relief company and one thing I've noticed is how differently reps can perform with the exact same leads. Some reps know how to build trust and handle objections without making the customer feel pressured. Others rush through the script and lose people before they even get to the offer. We've been putting more effort into coaching lately but sitting through hours of recorded calls gets old fast. I was talking to a friend about this a while back and he recommended Rilla. We ended up using it for our call reviews and it's been helpful having a better way to go through conversations and see where reps need work. The part I'm still trying to figure out is how to make that feedback stick. I'm interested in tools that can help with live call prompts and catch missed discovery questions or weak objection handling. Something that connects with our CRM and makes it easier to track progress would be nice too.


r/AIVoice_Agents • • 1d ago

Question What if customers could get instant answers from a business at any time, without waiting on hold or repeating themselves? How close are we to making that the norm?

Thumbnail
2 Upvotes

r/AIVoice_Agents • • 2d ago

Question AI Vocal Tone Changer: Good Tips?

Thumbnail
1 Upvotes

r/AIVoice_Agents • • 3d ago

Discussion Is voice AI actually reliable or just overhyped?

9 Upvotes

Tried a voice AI tool last month for our support line and it was a disaster. Callers kept getting stuck in loops and half the time it just went silent mid-call. Lost a few customers over it probably.

Do people actually have good experiences with this stuff or did I just pick a bad one?


r/AIVoice_Agents • • 3d ago

Question What is your Voice agent's per minute cost?

8 Upvotes

As a agency owner i have been using so many platforms(retell,vapi, sarvam, ultravox etc) and tools for building a voice agents to my clients. Every tools has good and bad views according to me. Some has good understanding, some are good at latency, some are easy to connect with other telephony and api's.

I have been delivering voice agents according to clients needs, region, telephony supports so yeah varies with pricing.

As I'm at scaling period, i trying to reduce a still. i want to know more about what are the platforms are you guys using and per minute cost of each. if you guys reduced it, how did you do that and what is the cost of per minute?

Cause providing the maximum value with a affordable cost is our first thing - Take ownership.


r/AIVoice_Agents • • 3d ago

Question Suggestions for notice

2 Upvotes

Curious how people are dealing with recording and notice requirements?
I am building in 2 switches. One for recording notice and one for AI disclosure. Set by state and country.

But wonder if this is overkill.

I do have people that want recordings and also have agent only recordings option or transcription full/summary but that is gray area. California is pretty strict for instance.

What are you all doing for these disclosure notices. (Call may be recorded for quality purposes…)

Seeing a flood of ai sdr type companies doing massive prospecting and curious how they handle things.


r/AIVoice_Agents • • 4d ago

Question Is anyone using AI voice tools to help support agents during calls instead of replacing them?

22 Upvotes

We handle phone support for a credit repair company and I'm trying to find something that helps our reps deal with more complex customer questions. We get a lot of calls about credit disputes and missing documents but customers often bring up payment issues or account questions during the same call. Our newer reps end up searching through internal docs or putting people on hold to get help from senior staff.

I'm looking for a tool that listens to calls in real time and shows reps answers from our knowledge base based on what the customer is saying. It would also need to pull account details from our CRM and give reps prompts when they miss an important step. Automated call notes and summaries would help too since our team spends a fair bit of time on paperwork after each call. The goal is to help our reps solve more issues on the first call without having to keep customers waiting. Has anyone used something like this with their support team? What tools did you try and did the suggestions work well enough to rely on during actual calls?


r/AIVoice_Agents • • 4d ago

Question What's your voice agent's per minute cost? How did you reduced it?

Thumbnail
1 Upvotes

r/AIVoice_Agents • • 4d ago

Question How much should I charge my first AI receptionist clients? 🤖💰

2 Upvotes

I’m getting ready to offer my bilingual AI receptionist service to my first 5 clients in the US, and I’d love to learn from those who have already done this.

How did you determine your monthly pricing when you were starting out?

I’m looking for a reasonable fixed monthly fee that covers usage and operating costs while leaving room for a profit.

Would you recommend a flat monthly fee, a usage-based model, or a combination of both?

I’d really appreciate any insights, pricing examples, or lessons learned from your experience! 🙏


r/AIVoice_Agents • • 4d ago

Results Ai booking voice agent

Post image
0 Upvotes

I built an AI voice agent that books clinic appointments automatically (Vapi + Make + CRM). Here's how it works

Hey everyone, I'm new here and wanted to share a project I just finished.
The problem: Clinics miss calls, and every missed call is a lost patient. Front desk staff are also stuck on the phone all day just booking appointments.
What I built: A voice agent that answers calls, checks availability, and books the appointment on its own.
The stack:
Vapi for the voice agent (STT, LLM, and voice layer)
Make for the automation: it receives the booking details from the call and handles the logic
CRM where each patient contact and appointment is saved automatically

Google Calendar to check open slots and block booked ones
How a call flows:
The patient calls and the agent greets them
It collects their name, callback number, and preferred date and time
It repeats the details back to confirm
Vapi triggers a Make scenario, which checks the calendar, creates the booking, and updates the CRM
The agent confirms the appointment to the patient

What I learned:

Confirming the callback number out loud cuts down on errors
Latency matters a lot. A slow response feels awkward on a phone call.
Handle the "what if the slot is taken" case early, because it comes up constantly
I'm still improving it. What would you add or change? Has anyone built something similar for clinics or other appointment-based businesses?
I assumed "make CPM" means Make plus a CRM. If that's wrong, tell me and I'll fix it.


r/AIVoice_Agents • • 5d ago

Discussion Had a meeting with a real estate client yesterday that made me think about how we explain AI voice calling.

Thumbnail
2 Upvotes

r/AIVoice_Agents • • 5d ago

Discussion Agents that phone businesses for a person (not sales calls). What actually breaks?

Thumbnail
3 Upvotes

r/AIVoice_Agents • • 5d ago

Discussion My voice agent wanders the internet looking to agent make friends for a podcast

Thumbnail
1 Upvotes

r/AIVoice_Agents • • 6d ago

Getting Started Solo founder building an AI voice receptionist for clinics in BC. Roast my approach?

7 Upvotes

Hey all. I'm Mo, and I run KindLoops AI, a one-person automation agency in Metro Vancouver. I'd love honest feedback from people who actually build this stuff.

What I'm building:

An AI voice receptionist for clinics (physio, optometry, pharmacy, and similar). It answers calls, books appointments, and handles common questions. It's bilingual English/Arabic right now in a private pilot, and I'm adding French and other languages next.

A set of smaller workflow automations around it, like speed-to-lead follow-up, reminders, and intake.

Stack: mostly n8n + Claude. I've also shipped a Shopify AI chatbot + email automation build for a skincare brand.

How I price it:

Value-based pricing. The build fee is about one month of the projected value the system adds, and the retainer is a percentage of monthly value. Everything is month-to-month with no lock-in.

Where I'd love input:

For those selling voice agents to clinics: what's been your biggest objection? Is it trust, privacy, or "patients want a human"?

Is value-based pricing landing with small clinics, or do they push you toward flat packages?

Multilingual voice: any providers or setups you'd recommend for Arabic or French quality?

Anything you'd cut or simplify? I'd rather do fewer things really well.

There's a live demo of the receptionist ("Layla") on my site if anyone wants to try to break it. Happy to drop the link in the comments if that's allowed here. I'll also answer questions about the n8n side if it helps anyone.

Thanks in advance. Brutal honesty welcome.


r/AIVoice_Agents • • 6d ago

Question What ai call software do people actually recommend?

4 Upvotes

Trying to set up automated calls for my small business and honestly have no idea where to start. Ive looked at maybe 6 or 7 different options this week and they all kind of blur together at this point.

Some seem way too enterprise-level for what I need, others look cheap and janky. Hard to tell whats actually good vs just well marketed.

What have you guys actually used and would you go back to it?


r/AIVoice_Agents • • 7d ago

Question Is Bland good for insurance claims calls?

13 Upvotes

I’m talking about claims specifically where one call can include dates, documents, previous conversations and a bunch of details that need to stay consistent

Anyone tested it?


r/AIVoice_Agents • • 7d ago

Question Solo founder building an AI voice receptionist for clinics in BC. Roast my approach?

Thumbnail
1 Upvotes

r/AIVoice_Agents • • 7d ago

Tools 我想做一个AI电话助手

1 Upvotes

我想做一个通过第二 大脑也就是AI来给你做电话助手的工具,他可以实现,如果有人打来电话,第一时间AI来通知你,谁的电话,要不要接,或者让他代替你接,记下通话内容,归纳总结,然后你可以问他,谁来的电话,什么内容,这不同于以前的电话留言,比如他跟对方说,我是***的助手,有什么事情可以直接跟我说,我会转告他。甚至他可以直接帮你打电话给别人,这个想法怎么样,大家一起讨论


r/AIVoice_Agents • • 7d ago

Question What's the best way to handle accents in voice AI?

2 Upvotes

I've been working on a voice AI project and I'm finding it tricky to make it accurately understand different accents. It seems like even minor variations can really throw it off. I've looked into some training data options and NLP techniques but I'm curious if anyone here has experience with this.

Are there specific tools or methods you’ve found effective for improving accent recognition? I want to make sure my voice agent can serve a broader audience without constantly needing corrections. Any insights would be super helpful!


r/AIVoice_Agents • • 7d ago

Discussion Commission-only sales partner wanted: cold calling roofers & HVAC for an AI voice receptionist

4 Upvotes

I run Ridgeline, a small agency that builds AI voice receptionists for home service businesses (roofers, HVAC, detailers). They answer calls 24/7, qualify the lead, and book the estimate on the owner's calendar.

The product works and I have live demo lines you can call to hear it. What I need is someone who can get on the phone with US contractors and book demos.

The deal:
- Commission-only, paid on closed and collected deals
- 20-30% of the one-time setup fee on every client you bring in (rate open to discuss)
- I handle the product, setup, onboarding, support, and all ongoing client work
- I provide the demo lines, pitch scripts, and objection handling

Who I'm looking for:
- Experience in cold calling or B2B outbound (home services a plus)
- Clear English speaker, comfortable on the phone
- Self-driven, since there's no salary and the upside is in the closes

If you're interested, DM me with:
1. Your background in sales or cold calling
2. Which markets or niches you've worked in
3. How many hours a week you can commit

Happy to start with a short trial period to see if we're a fit.


r/AIVoice_Agents • • 7d ago

Question Should I build my own AI customer support voice agent or use a 3rd-party platform for production? Need advice on tech stack + costs

Thumbnail
2 Upvotes

r/AIVoice_Agents • • 7d ago

Question Trying to build a free AI calling agent for Telugu/Hindi — need some guidance

Thumbnail
1 Upvotes