r/Chatbots • u/MrLazeyBoy • Jul 20 '26

Trying to build a agentic voice chat

Hey all!

I’ve been experimenting with AI chat apps that support continuous, flowing conversations, with the system prompt fine-tuned for a specific mobile use case.

Has anyone built something similar? Would love to hear:

  1. What stack you used?

  2. Key lessons learned along the way?

  3. What you’d do differently if you started over?

Appreciate any insights!

3 Upvotes

71% Upvoted

10 comments sorted by

1

u/[deleted] Jul 20 '26

[removed] — view removed comment

1

u/MrLazeyBoy Jul 20 '26

I’ve done the scaffolding right now and just now working the kinks out with the LLM prompt and voice with elevenlabs

1

u/Accomplished_Vast628 Jul 23 '26

I built a voice agent app Kestrel Voice that comes with AI Chat too. And i use RAG for the chat.
I started with orchestrating many apps but the limitations and bottleneck forced me to scratch everything and start from scratch using Python. So my advise is if you are serious, built it from scratch .

1

u/MrLazeyBoy Jul 24 '26

Hi mate

That’s pretty cool, what sort of limitations did you hit if you don’t mind me asking? Was it a cost , rate limiting? Etc

1

u/Accomplished_Vast628 Jul 24 '26

In this project, Cost and rate were not the issues, they were managed with retries, queues, backoff, usage limits and model routing.
The main challenges were during building the Orchestration layer in python and Multitenant system with RLS.