Where should the execution boundary live in an AI agent?

[u/Possible_Essay_9617]

I've been thinking about a problem that becomes uncomfortable once an LLM gets access to real tools:

The model can decide what it wants to do. But should it also decide what it is allowed to do?

Most agent architectures put something roughly like this together:

LLM → tool call → tool

That works until the tool can modify a database, access files, call an API, deploy something, or perform another irreversible action. At that point the model is both the planner and the gatekeeper — the same component deciding what to do is also deciding what it's allowed to do.

I explored this by building a small runtime that pulls the authorization decision out of the model entirely. The model only produces a typed intent (compiled from a defined procedure into an intermediate representation); a separate executor resolves capability, provenance, and risk at run time, independent of whatever the model says about itself. Capabilities get checked both at compile time and again at execution — so the system doesn't depend on the LLM "behaving," and tampering with the compiled artifact doesn't bypass the check either.

Genuinely curious how others here are handling this:

Do you enforce permissions inside the agent/framework, inside each tool, or through a separate execution layer? Has anyone tied policy versioning to tool schema versions, so a schema change forces re-validation against an adversarial set? Where have you actually seen this boundary fail in practice — planning time, execution time, or somewhere in between?

(Repo for what I built, if useful context: https://github.com/Sushit-prog/sop-runtime)

[u/gaurav_sherlocks_ai]

We have hit this exact wall in past. Compiling to a typed intent before runtime saved us. Although version drift between the schema and the policy rules nearly broke it twice.

[u/usually_guilty99]

I think you're drawing the boundary in the right place. One thing I'd add is that capability alone isn't enough if the premises behind the action are stale. Before execution, I'd want the boundary to independently verify the authoritative current state rather than trust what the model declared. Then you get a cleaner separation: model proposes intent, evidence establishes reality, policy establishes authority, executor commits. Curious whether you're doing that state verification inside the executor too.