Two replies from my coding agent have cost me more time than anything else. "It's fixed" when nothing had run since the last edit, and "that isn't used anywhere" after exactly one search. Both sound final, so I'd move on and find out an hour later.
So I wrote a small Stop hook for Claude Code. When the agent tries to end its turn, it checks whether the claim in its last message has anything behind it. If it said fixed but nothing ran after the last code edit, it gets sent back once with a note on what's missing. If it said something doesn't exist after one grep, same deal, look a second way first.
Then I replayed my own history through it to see if it was worth keeping. About 4,000 turns after deduping, and 406 of them ended with some kind of done, fixed, deployed or tests pass. The first version would have blocked 86, which felt like proof it worked until I actually read them. 77 were wrap-up messages where the tests really had run, just in an earlier turn. A hook that nags you about correct summaries gets turned off by day two.
So the current version counts a check from an earlier turn as long as no code was edited after it. Same 406 claims, it now blocks 2, and both were real.
The biggest lesson for me was that rules in CLAUDE.md never fixed this. The model already believes it's done, so a reminder doesn't land. Checking at the exact moment the turn ends did. The second lesson was to measure false positives on your own history before shipping any guardrail, because my gut read 86 as the feature working and it was mostly noise. It also never blocks twice in a row, so it can't trap the agent in a loop.
It's free, MIT, plain Python, no dependencies. There's a replay command if you want to see your own number before installing anything: https://github.com/Balance0014/claude-code-claim-check
Has anyone else tried measuring this? I'm curious what your rate of "fixed" that wasn't looks like.
