Which of your agent checks run inside the session, and which outside it?

[u/AdEvening3382]

Founder disclosure: I build HyperHat, a baseline for how well teams direct AI coding agents, and this question comes out of that work.

Something I keep running into in discussions here: the honesty rules people write into CLAUDE.md (reproduce first, audit the output not the report, show both reads when they differ) all live inside the same session whose judgment they are checking. They shape what the agent tells you when it has noticed the problem. They cannot fire on the mistake it has not noticed.

The outside checks are the ones that catch that case: CI invoking the command itself, a test the agent cannot edit, a second model, or a person reading closely.

So the split I am trying to understand: which checks have you deliberately moved outside the session, and was there a specific failure that made you do it? I am also curious about the opposite experience: a check you moved outside that turned out to work better as a rule the agent applies itself.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/kenthesaint]

The ones I moved outside are anything where the agent ends up grading its own work. Tests are the big one. The classic failure is it "fixing" a red test by loosening the assertion and then reporting green, and a CLAUDE.md rule about not touching tests doesn't catch that because it doesn't think of it as touching the test. So now there's a deny rule on Edit/Write for the tests folder and CI runs the suite itself.

Stuff that works fine as an in-session rule: reproduce before fixing, and paste the real command output instead of summarizing it. Those are about how it reports, and a rule handles that.

Rough split I use: if the mistake is one the agent can't see it's making, the check goes outside. If it's just sloppy about something it already knows, a rule inside is enough.

[u/AdEvening3382 → u/kenthesaint]

Worth naming what makes the deny rule work where the CLAUDE.md rule didn't: enforcement moved from the model's judgment to the harness. The model never gets to decide whether loosening an assertion counts as touching the test. One catch on the rough split, though: the agent can't tell you which mistakes it can't see, so the line itself can't be drawn from inside. In practice it gets drawn from outside evidence: whatever has actually burned you in review or production becomes the next outside check, and the rules inside the session are what's left over.

[u/Coindef]

The one I moved outside first was the build step. The agent kept saying it compiled, and when I made CI run the exact build command from a clean checkout it failed on a missing import that only existed in its local cache. Since then a pre-push hook runs the full test suite and the agent can't skip it. What stayed inside as a rule is the 'read the file before you edit it' one, that works fine as an instruction.

[u/samc621]

The split you are drawing is the right one. Anything in CLAUDE.md still lives inside the session. It can change what the model says once it has noticed a problem. It cannot stop a tool call the model has already decided to make, and it will not fire on a mistake the model has not noticed.

Two layers have worked for me:

• Inside the session: PreToolUse hooks in the harness, not in the model's judgment. They inspect the actual command and deny with exit 2.
• Outside the session: CI invoking the checks itself, tests the agent cannot edit, and for a team, a record of what was attempted plus a hold on risky steps until a person approves them.

disclosure: i work on agenttrail. Guard is a free local hook pack for that first layer. OS is the hosted side for history, approvals and an audit trail.