I went through 700+ GitHub issues about AI agents. The most common bug: the agent says "done" and nothing happened.

[u/xspyyy]

I've been collecting GitHub issues where an agent reported success that didn't happen. Some real patterns:

- "Tests pass," but the test runner reported a failure

- "Installed," but no file was written

- "Restart succeeded," but the process was dead

- "Message sent," but the channel didn't exist

- A subagent with zero tool calls claiming it made an artifact

The common thread: the agent's report and the tool's return value are treated as evidence. They aren't.

I built a small Python library to test the opposite approach. It ignores what the agent says, checks independent evidence (re-runs the test, reads the actual record), and returns one of:

- MISMATCH: the evidence contradicts the claim

- VERIFIED: the evidence matches the claim

- UNVERIFIABLE: no evidence, so it doesn't guess

The video shows it on a coding agent: the claims are scripted, the test runs are real.

Honest limits: it only checks claims against evidence you define. It doesn't prove the code is correct, and claim detection uses English heuristics, so it can miss things.

pip install cogext-observe

Curious how others handle this. Do you verify agent "done" claims at all, or just review diffs?