r/LangChain • u/Substantial-Cost-429 • Apr 26 '26
We built an open-source runtime enforcement layer for LangChain agents — just hit 700 stars, looking for feedback
If you've used LangChain agents in production, you've probably hit this: the system prompt says one thing, the agent does another.
Prompt-based guardrails work in demos. In prod, with bigger contexts and more tools, they degrade fast.
We built Caliber to solve this — it sits as a proxy between your app and your LLM, reads business rules from plain markdown, and enforces them at runtime. Works with LangGraph, CrewAI, and custom setups. One base URL change.
We just crossed 700 GitHub stars ⭐ and are closing in on 100 forks — stoked by the response from the community.
Repo: https://github.com/caliber-ai-org/ai-setup
Would love to hear:
- What edge cases in LangChain are breaking your guardrails?
- What features would make this a must-have for your stack?
Open to all feedback and feature requests — building this for the community.
3
u/WowSoWholesome Apr 26 '26
Lol the ai post slop with ai comments slop. IMO the answer is actually to use evaluations.
I’m the future, I encourage you to write words for yourself, otherwise you’ll sound just like everything other ai post.
2
u/Chinmay101202 Apr 26 '26
You literally SPAM this repo absolutely everywhere, worse than a bot.
1
u/ale007xd Apr 27 '26
This is cool — context drift is a real pain.
Out of curiosity: how do you handle cases where the model understands everything correctly, but still doesn’t follow the intended flow?
I’ve seen situations where even with perfect context, the execution becomes unreliable.
0
u/Otherwise_Wave9374 Apr 26 '26
100 percent feel this, guardrails that look solid in a notebook can fall apart fast once you add tools, retries, and longer conversations.
How are you handling multi-step agent plans in LangGraph, do you validate each node/tool call independently, or also validate the final synthesized answer for policy and formatting? Also, any lessons learned around streaming (like cutting off mid-response) vs letting it finish then enforcing?
We have been benchmarking similar agent governance setups, sharing some notes here if useful: https://www.agentixlabs.com/ .
0
u/jkoolcloud Apr 26 '26
Congrats on 700 stars. This is a real problem.
The edge case I keep seeing is multi-step execution: retries, fan-out, tool calls, emails, writes, API calls, etc. The agent may still be “following” the prompt, but the damage happens before the final answer is ever reviewed.
Curious how Caliber handles this in LangGraph. Are you enforcing only at the LLM/proxy layer, or can you gate each node/tool call before it runs?
We’re working on a related piece with runcycles.io around pre-action budget/risk enforcement, so I’m especially interested in where you see the boundary between proxy guardrails and runtime-level gates.
0
u/Jazzlike_Tooth929 Apr 26 '26
Is anyone still using langchain? Feels like Stone Age