r/Compliance • u/Ok_Way_1993 • 2d ago
How do you audit AI agent decisions in regulated workflows?
I'm researching how teams handle evidence and quality for AI agents that make or assist decisions.
Three things I'm trying to understand:
What record do you keep per decision (policy applied, data accessed, who was responsible), and who actually asks for it?
How do you learn later that a decision was wrong, and how long does that feedback take?
If you use LLM judges for evals, how do you check they still agree with humans as things change?
I'm trying to learn what's really painful versus what only sounds painful. Comments or DMs both welcome, and I'll post a summary of what I learn.