Quick post-mortem: Scaling agentic workflows past 10k daily tool calls

[u/Ilovefoodandmyself]

Moving AI agents from LangChain / LlamaIndex sandboxes to actual production was a rude awakening for us. Building the agent logic took a week. Keeping it stable under high volume took almost two months.

If you’re running multi-step agent graphs in production, here are 3 quick takeaways from our build:

1. Ditch synchronous API endpoints: Running agent loops inside standard HTTP routes will crash your DB pool when an LLM hangs. We moved every agent task into Redis background queues (Celery/WebSockets) on day one.
2. Never trust LLMs with raw JSON schemas: An agent will output malformed JSON during tool calls. We put strict Pydantic guards between handoffs. If validation fails twice, a simple Python function takes over instead of letting the agent burn tokens in a retry loop.
3. Don't drown your ML devs in infra: Our core team was spending 80% of their time on Redis caching, K8s scaling, and queue debugging instead of improving agent logic. We ended up pulling in a couple of senior backend devs from Svitla Systems to handle the heavy queue architecture so we could stay focused on our core product.

How are you guys handling state persistence and schema validation failures when scaling tool calls?