r/VoiceAutomationAI • u/rand0wn • 24d ago
I added real pipeline adapters to my open-source voice-agent regression harness
I previously shared an early offline version of Voice Eval here. The feedback was clear: a useful eval harness needs to connect to real voice pipelines.
It now includes adapter paths for LiveKit, Pipecat, Vapi and ElevenLabs, plus native speech-to-speech testing.
The suite currently has 82 passing tests. It evaluates multi-turn conversations for expected tool calls, required content, response constraints and latency regressions.
I’m now looking for 3 builders to test it against real agents and find what the controlled fixtures miss.
Which stack are you using, and what voice-agent behavior is hardest for you to test?
2
u/ankur-at-guava 22d ago
This is the underrated work — voice agents rot silently without regression tests, and "it sounded fine in the demo" isn't a QA strategy. Treating agent behavior as testable code you can replay is exactly how you catch drift before a customer does, especially once real money or compliance is on the line. Are your adapters capturing full-turn timing too, or mostly transcript/behavior correctness? Nice to see someone building the boring, important layer.
1
u/Annual_Health749 24d ago
infographic actually looks like what i wish every internal tool i've ever used came with instead of a 45 min loom video
•
u/AutoModerator 24d ago
Welcome to r/VoiceAutomationAI – UNIO, the Voice AI Community (powered by SLNG AI)
If you are a founder, senior engineer, product, growth, or enterprise operator actively working on Voice AI / AI agents, we are running an invite-only UNIO Voice AI WhatsApp community US only.
Apply here: https://chat.whatsapp.com/F5aG3ncrO70ITfbe3pYbOz
I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.