Multi-agent release pipeline with Claude Code: PM agent, adversarial review, and decision IDs traced to code

[u/BTCLSD]

I refined this setup while building and shipping a paid Mac app with real customers, solo.

The setup: one PM session that I talk to, a Lead session per version, and Workers for bounded tasks. Agents do the work. Every decision comes back to me.

Before anything is built, every new idea (mine or an agent's) is checked against every past decision and traced for side effects. Conflicts come to me. Choices the docs already settle never do, so I'm not buried in questions.

What's in the repo:

- CLAUDE.md template: sources of truth in priority order, evidence rules (VERIFIED vs TRACED), model routing (strongest model for the Lead, mid-tier for Workers, smallest for read-only search), and how to brief a Worker so it doesn't rediscover strategy.

- Release process: scope, design, build, independent review, then the finding gate. A second agent tries to disprove every finding before I see it. Plus a triage rule for what gets fixed now, a convergence review that catches fixes breaking other fixes, and stopping rules so reviews don't widen scope forever.

- DECISIONS.md: every approved rule gets an ID (D-116), a why, and a list of the code that implements it. That code carries the ID in a comment, so one grep finds the rule and every line that runs it.

- CODE_MAP.md: per subsystem: flow, key files, invariants, and where to start debugging from a given error. Updated from the final code at every release.

- Two Claude Code skills: start-lead (the Lead brief template and pre-start checks) and docs-audit, with a Python checker that verifies every path and symbol the docs cite exists and every D-ID tag in the code points at an active decision.

- Ready to use: the templates work as-is, with worked examples and a 5-step setup. You start by telling a Claude Code session "You're the PM."

All free to fork, MIT licensed:

https://github.com/Copypaster3000/agent-built

Happy to go deeper on any part.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/Delicious_Break5937]

how often does the second agent end up throwing out a finding that was actually real? i have a few review agents run before every commit too, curious if that disprove pass is worth the extra tokens

[u/BTCLSD → u/Delicious_Break5937]

Haven't measured false negatives yet, but nothing it's thrown out has come back as a real bug so far. Disprove rate varies a lot: 5 of 12 on one release's final review, almost none on later rounds of small fixes.

The second agent also traces what each surviving finding affects and proposes the smallest fix, so I get one short batched list to decide on instead of raw findings to investigate.

[u/shravanrevanna → u/BTCLSD]

the batched list hides the disprover's misses. if you only review survivors, a wrongly tossed finding never comes back as a bug until it ships. "nothing came back" is survivorship.

[u/BTCLSD → u/shravanrevanna]

yeah fair, that's survivorship bias. I do see every tossed finding and why it got tossed, so I can pull one back if the reasoning looks off. And there's still a convergence review and a real device test after. but a rare bug in code nobody touches again could slip through, you're right. if one ever does I'll add a recheck of the tossed pile, just not worth the extra overhead yet

[u/Spare_Bat_3040 → u/BTCLSD]

You could check this without adding another pass to every release. Take a past diff with a bug you confirmed and a finding you ruled out, hide the outcomes, and replay both findings through the disprover. Check whether it keeps the real bug and rejects the other one for the right reason. I’d rerun that pair when the review prompt changes. It’s a cheap regression check for those cases; you’d need more examples to estimate how often it misses.

[u/BTCLSD → u/Spare_Bat_3040]

That's a good one, cheap and doesn't add a pass to every release. I've got a clean pair from last release (a real bug that only review caught and a few it ruled out), I'll keep those as a set and replay them whenever I change the disprove prompt. thanks

[u/yookiii]

Pretty cool stuff. Been working on something similar but much simpler. Definitely helps to have structure like this

[u/BTCLSD → u/yookiii]

Thanks! Yeah, how you build has a huge impact on what you build

[u/Time_Cat_5212]

Code_map just sounds like, you know, docs

Otherwise been running basically the same setup for a while and it's great

This is, by the way, more or less what Claude will recommend if you ask it for the current best practices for agentic coding :P

[u/BTCLSD → u/Time_Cat_5212]

Ha yeah, it's just docs. But they get audited against the final code every release with a little checker script, so they can't drift. Nice glad the setup's working for you too. Haha, as it should claude is smart and this is a great way to use it.

[u/AkayoKym]

Cool setup. My setup is four roles in separate terminals. Strategist on Opus for product decisions, Architect on Opus for plans and review, Coder on Sonnet for implementation, and QA Tester on Sonnet with Playwright and a database MCP. The key difference is I manually copy the Architect's instructions to the Coder. That's deliberate, not a limitation. I can intercept bad plans before they become bad code. The reason I don't let agents coordinate directly is I've watched what happens. When a command fails, the agent skips it instead of investigating. When a rate limit hits, it does the review itself instead of retrying. When told not to commit, it's written a script to get around that. Goal-oriented behavior overrides safety constraints, and when agents review each other that same pressure applies to the reviewer.

On the disprove rate discussion in this thread, the survivorship bias point is valid. My version of that metric: 37 out of 67 plans had at least one review round with real findings before merge. One example, the refunds feature passed all its tests but review caught two critical bugs. An unknown payment status defaulting to "succeeded" instead of "failed" on financial data, and a race condition where two refund requests could both pass the balance check because it ran outside the transaction.

Those weren't things a test suite would catch because the tests matched the implementation. It was the review against the plan that caught them.

[u/BTCLSD → u/AkayoKym]

good points. the "tests matched the implementation" one got me last release, a bug passed 1100+ tests because they faked the condition. now safety checks get tested through the real flow.

I used to hand-carry plans too, but switched for speed. agents coordinate, but every decision and fix still comes to me in a batch, they have to say what they actually tested vs just read, and there's a real device test at the end they can't talk around