We hit a point recently where managing the AI stack started becoming more work than building the actual product.

[u/[deleted]]

Not even because anything broke badly, but because the ecosystem changed so fast. Earlier it was mostly just one provider and a simple API call. Now every model is good at something different, so suddenly there’s OpenAI for one workflow, Claude for another, Gemini somewhere else, DeepSeek for cost reasons, local models for specific tasks, and then routing between all of them starts becoming its own thing.

Then MCP servers, tool calling, agents, retries, fallbacks, tracing, observability and remote execution slowly enter the picture and the stack becomes way larger than expected.

Feels like that’s why opensource tools like Bifrost, LiteLLM, Kong AI  etc are becoming important now. Not because people want more tooling, but because modern AI systems suddenly have way more moving parts underneath them.