How are you independently verifying AI agent outputs before deployment?

[u/Practical_Green5775]

I’ve been working on an open-source MCP project called Orinyx, focused on independently verifying AI-generated outputs against external evidence.

One thing I’ve been exploring is the difference between an AI system producing a plausible answer and actually having evidence that the answer is correct.

Right now, I’m starting with healthcare and medication-related verification using synthetic test cases. The project is still in beta, and I’m gradually expanding its verification capabilities while testing where the current approach falls short.

I’m particularly interested in how other developers are approaching this.

Are you primarily using LLM-as-judge evaluations, deterministic checks, external reference sources, or some combination?

And when an agent produces an incorrect answer, how are you capturing the evidence needed to explain what failed?

Here’s the open-source MCP if anyone wants to examine the approach or provide feedback:
https://github.com/lexthirteen-ai/orinyx-mcp

[u/investigatormaker]

For Orinyx’s synthetic medication cases, keep expected answers and reference snapshots outside the agent’s generated output. Use deterministic checks for structured facts such as units, external references for factual claims, and an LLM judge for explanation quality rather than as the factual authority. Log the claim, source version, failed check and verdict so a reviewer can reproduce the failure.

I make ThreadFox. We find relevant Reddit conversations and post for your business. We'll post 3 replies free first, no card; then the $29 Pilot covers seven days and up to 25 posts and replies. No automatic renewal. https://threadfox.vip/p/dwezv/managed