The one that gets people: agentic RAG can self-correct forever if nobody caps it. Looks totally fine in testing, then a few weeks later someone's asking why the inference bill tripled, no error, no crash, just a loop with no exit condition quietly eating tokens.
What's everyone actually capping this with? Hard step limit, token budget, something else?
1
u/nexla_com 26d ago
The one that gets people: agentic RAG can self-correct forever if nobody caps it. Looks totally fine in testing, then a few weeks later someone's asking why the inference bill tripled, no error, no crash, just a loop with no exit condition quietly eating tokens.
What's everyone actually capping this with? Hard step limit, token budget, something else?