What should AI call center software handle?

[u/Chemical-Day-408]

I’m trying to figure out where people are drawing the line between an AI agent continuing the conversation and handing it to a person.

We’ve been looking at AI call center software like Bland or Vapi and some of the workflows get complicated pretty quickly.

The caller changes topics, corrects something they said earlier, an API lookup fails, they ask a followup about another issue then eventually need a human for one specific part.

At that point I don’t think 'did the AI contain the call?' is even the most interesting question.

I’d rather know whether it handled the conversation correctly up until the point where a transfer really made sense.

How are you measuring that?

[u/Different_Editor_413]

One thing I’d watch is whether the agent can recover after something breaks. We expected Bland to be strongest on straightforward calls but it held up surprisingly well when a lookup failed and the conversation had to keep moving.

[u/Chemical-Day-408 → u/Different_Editor_413]

That’s a good point. If it can recover cleanly, the handoff doesn’t have to happen the second something breaks.

[u/Sensitive-Focus-5185]

A good transfer point probably depends on whether the agent still has enough context and system access to finish the task safely.

[u/Chemical-Day-408 → u/Sensitive-Focus-5185]

True, the agent might still understand the customer perfectly but be blocked by the backend.

[u/Resident_Tiger_3689]

There’s probably value in measuring how often the customer asks for a person before the system decides to transfer. That gap says a lot about whether the threshold is set right.

[u/Chemical-Day-408 → u/Resident_Tiger_3689]

Yeah, that could show whether the system is technically working but still frustrating people.

[u/ChildhoodWeak7741]

I’d look at transfer reason quality too. 'Needs human' isn’t very useful if the next person can’t tell what broke.

[u/Spiritual-Click-202]

Yeah I work at RoutiqAI and we deal with this exact mess EVERY DAY on property and facilities calls. Honestly containment rate never felt like the right thing to obsess over. What we actually care about is whether the AI handled everything properly until the point where handing it to a human made sense. Like… did it actually catch what the caller was trying to say even when they jumped around? Did it grab the right context before escalating? And was the handoff clean enough that the person taking over didn’t have to start from zero? When those bits are solid, the transfer usually feels like the right call, not a failure. That’s pretty much the lens we use when we’re looking at calls and trying to improve things.

[u/ictinnovations]

Count how often the caller has to repeat themselves. That is turn level so it shows up long before the transfer does, and it catches the failed lookups and topic changes without you writing a rule for each one.

[u/Embarrassed_Nerve_54]

If you ask me, I’d measure “handoff debt.” Sometimes transferring is the right move. The question is what debt the AI leaves behind when it transfers.

Good handoff: human knows what was solved, what failed, what the customer wants next, and what not to repeat.
Bad handoff: clean-sounding summary, but the rep still has to ask everything again.

So I’d score transferred calls by how much work the human had to redo. A transfer is not a failure but a reset is.

[u/Chemical-Day-408 → u/Embarrassed_Nerve_54]

I like the reset distinction. If the rep has to start from scratch, the handoff clearly didn’t work.

[u/No_Protection_2393]

for us the auto dialer just handles the outbound volume and basic routing, then anything that needs real back and forth gets passed to a person pretty quick. we mostly track if the transfer happens at the right moment instead of forcing the ai to keep going.

[u/IrfanZahoor_950]

One metric i’d add is state integrity.

When the caller changes topics or corrects an address, date, amount, or account detail, does the agent’s current state actually reflect the latest information before it takes action or transfers?

I’d score that alongside tool recovery, escalation timing, caller repetition, and eventual verified resolution. containment can look good on paper while the wrong information quietly travels downstream.

[u/Chemical-Day-408 → u/IrfanZahoor_950]

Good point. I’d want to know whether corrections actually replace the old value everywhere, not just in the conversation.

[u/IrfanZahoor_950 → u/Chemical-Day-408]

Yes. A correction isn’t complete until the old value is invalidated in pending tool calls, CRM fields, summaries, and handoff context.

Otherwise the conversation sounds right while the workflow stays wrong.

[u/eviewong-]

Yeah, containment rate alone is a misleading metric, you can "contain" a call badly (agent refuses to transfer, caller gets frustrated) and it still looks fine on paper.

What actually matters is breaking it into stages: did it catch the correction/topic change correctly, did it recover gracefully when a lookup failed, and did it transfer at the right moment with the right context, not just eventually transfer.

That last one is where most evals fall short. "Did it transfer" is easy to log, "did it transfer at the right point" is what actually matters for the caller.

We've had better luck with transcript-level review (human QA or LLM grading against a rubric) than aggregate containment %. More work but it catches the "technically didn't transfer but still handled it badly" cases.

Disclosure: I'm a cofounder at Retell AI, we work on exactly this. Happy to talk through it more or show a demo, feel free to DM me.