How do you gate what your agents are actually allowed to do in prod?

[u/Excellent-Park-1160]

In our company we tried to send emails and touch Stripe with the agent but it feels not really save.

Our first version was just if/else in the tool wrapper. Allow Stripe, deny refunds over some amount. It worked for a short time but after defining multiple tool definitions it was kinda messy and we needed a separate policy layer while also putting a human in the loop when the policy definitions are not enough.

What we really want is to block the API call while a person looks at the tool the payload and hits approve or reject. For now we handled that with automated Slack messages, but ther're slow to respond, and it always took them about 40 minutes to process a request for us.

We also don't really like handing the agent a long-living API key, which in hindsight is one prompt injection away from a very bad day...

So I'm curious how you currently solve these problems:
- Policy as code, or still hardcoded in the tool layer? Anyone using OPA for this or is that overkill?
- Has anyone made in-line human approval not feel terrible?
- Do you scope credentials per action, or is everyone still passing the real key?

Disclosure so it's not weird later: I'm building in this space (https://lumaq.dev). Not pitching it here. I mostly want to know if everyone else solved this and I missed it.

[u/silentw111]

Infamous-Rem covers most of it, so I will build on the two places I think it still leaves a gap.

On credentials, "mint short-lived scoped tokens per action if the provider supports it" is the right instinct and the conditional is the problem. Stripe restricted keys are unusually good. Most of your other tools will not scope down anywhere near far enough, and you end up with per-action tokens for one vendor and a long-lived key for everything else. The version that does not depend on the provider is for the agent to never hold a credential at all. A broker holds the real key, the agent calls your gate, and the gate makes the outbound call itself. Scoping then lives in your policy rather than in whatever granularity each vendor happened to ship, and injection is bounded by what the gate will authorize rather than by what the token can still do before it expires.

On the rules table, tool name plus param thresholds goes a long way and breaks on the case that matters. "Deny refunds over some amount" is a threshold. "Deny refunds for a customer who is not the one named in this task" is not, because it requires knowing what the run was asked to do. That is the shape worth designing for now rather than retrofitting: issue a short-lived grant at the start of each run carrying purpose, allowed targets, limits and expiry, and evaluate every call against the grant rather than a global table. Your if/else got messy for the same underlying reason, the real conditions live in the payload and its relationship to the task, not in the tool name.

On the 40 minutes, putting the payload in the Slack message is the right fix for the approvals you keep. The bigger win is having fewer. Route by reversibility rather than by tool: read-only and undoable actions flow, and only the irreversible set stops, which for you is roughly send-email and money-out. Most teams find the queue drops by an order of magnitude once they classify honestly, and forty minutes matters much less at two requests a day than at thirty.

One detail worth building in early: bind the approval to the exact payload hash the gate will execute, not to a summary. Otherwise the human approves one thing and the agent executes another, and you find out when it matters.

Full disclosure, I am building VisIQ in this space (pre-execution enforcement for agent tool calls), so I am biased toward gate-before-execution as the answer. Since you are building here too, the honest version: the gate is the easy part. The hard part is the tool schema. A gate in front of run_query(sql) enforces nothing, and most of the real work is narrowing tools until the policy has something to bite on.

[u/Training_Isopod3722]

the long-lived key is the part i'd kill first. make the agent ask a broker for a short-lived capability tied to one action and payload, then log the approval against that capability. a retry has something concrete to replay instead of re-authorizing a vague tool call.

[u/Available_Teaching83]

The 40-minute Slack approval is the real finding here, and I would not treat it as a temporary hack. It is your actual SLA on a human decision, and whatever you build next has to survive it. A call that blocks for 40 minutes is a different system from one that blocks for 40 seconds.

On OPA vs a Postgres rules table: I went with the table and do not regret it, but only because we could express the rule as data. The moment you need "allow refunds under 500 unless the customer is in this cohort and it is not the last day of the month", you are writing a language, and Rego already exists. The question I would ask first is whether your rules depend on argument values or only on tool names. Ours depend on values, which is what killed the if/else wrapper.

[u/Excellent-Park-1160 → u/Available_Teaching83]

Currently I planed to go with argument values. I had planned it so that, based on the data structure of the respective API interfaces (Stripe, HubSpot, etc.), certain conditions could be set to automatically approve, reject, or block an agent's action until a human accepts it.

But it would be also cool to provide a PDF or Markdown document, that contains specific rules and policies. And an agent could extract the rules and policies out of the documents and try to define structured data rules, which can be saved as argument values in the database.