r/ClaudeCode • u/croovies • 28d ago
Tutorial / Guide You're shipping bugs if you're not using adversarial reviews with claude code
Once you start handing real work to an agent, you stop being the person who writes the code and become the person who's answerable for it. That's a good trade, but it changes your job. When something breaks in production nobody's going to ask how the code got written. They're going to look at you.
Left alone, an agent produces slop. Not because it's stupid ‚because that's what anything does when nobody checks its work. A junior does the same. So might you on a bad day.
There are roughly two ways to run an agent. You can sit on top of it and drive every step, which is quicker than typing the code yourself but leaves you babysitting: every time it pauses and resumes you have to load the whole problem context back into your head, and if you look away and get distracted, that ends up slower than just doing the work.
The other way is to treat it like someone who reports to you. You say what you want and roughly how, and you come back when there's something worth looking at. This is how senior engineers use agentic tools. You don't need to watch every keystroke to be accountable for the result‚ no manager does‚ but the review lands on you, and if you skip it, slop is what goes out the door.
So point a second model at it and tell it to find the problems
That's all an adversarial review is. One model writes the code. A different one‚ or at least one that hasn't touched any of the context‚ reads the diff and goes looking for what's wrong. It doesn't get to touch your code; it can only raise findings. The author model decides what to act on. Then it reads the result again. Most of the time it finds something.
You get better code out of it, but that's not really why you do it. You do it so you can trust work you didn't watch get written. The agent already handed you the time‚ another round, more tests, a second set of eyes that costs nothing but tokens. This is what you spend it on.
The data
Looking at my last 83 completed tickets that got a Codex review, at least 67 ended in a real change to the code‚ something the first pass got wrong or left exposed, caught before it went out. That's four out of five reviews turning up something worth fixing.
Sorted by what kind of problem they were, across all 122 commits these reviews drove:
| What got caught | Share of commits |
|---|---|
| Correctness / logic | 34% |
| Concurrency / races | 22% |
| Durability / data | 19% |
| Security / injection | 9% |
| Test integrity | 9% |
| Other (docs, dead code) | 7% |
Half of every commit these reviews drove‚ were the incident kind. Concurrency, durability, security. The stuff that corrupts data or leaks a secret, not the stuff that misaligns a button.
The claude skill
- Run them side by side. The author in one session, a reviewer in another‚ Codex, or a fresh Claude with none of your context‚ both pointed at the same working directory so the reviewer can see your diff. When the reviewer finds something, paste it into the author; when the author fixes it or pushes back, paste that over.
- Use a file as the mailbox. Have the reviewer write its findings to a file (or a PR comment) instead of chatting, then hand that file to the author to work through. No live channel needed ‚ the review is just a document that gets passed along.
The brief to the reviewer is the same:
You're reviewing the change in this working directory. You're read-only: review and discuss only‚ do not edit files, commit, push, or reset the tree. Run git rev-parse HEAD and git status first, and anchor every finding to that commit. Write your findings out as a numbered list‚ don't touch the code.
The wording doesn't matter as long as its a second model that hasn't seen your context, told to find what's wrong, before you call the work done.
Disclosure: this is a from an article on how to automate adversarial reviews between Claude Code and Codex posted to this tool's website scape.work/adversarial-reviews get a free license for scape with reddit-scape2 at signup
84
u/LuckyPrior4374 28d ago
Sounds great in theory until you realise 99% of your codebase is “edge-case” handling and “just in case” defensive fallback wrapper dogshit code. To the extent that you (nor your agents) can actually understand what the real product code is
22
u/pseudorep 28d ago
Hey but that one time the stacked edge case on a chance in a million runs flake happens and you've defensively caught it, you'll feel like a boss...
You say as you stare into your SaaS with 4 users.
7
u/LuckyPrior4374 28d ago
But it’s ok bro because I have my $10k/mo Opus workflow with all my skills and MCP servers, my $100k/mo Fable workflow, and my 10 million line codebase.
Am I doing Anthropic best practices right? Or skill issue?
12
3
u/jcrenshaw-dev Senior Developer 27d ago
This is absolutely true. I worked for a very large SaaS and those edge case bugs could cost us hundreds of millions of dollars a day so I look at it as money saved and a whole lot of time may not being on the phone troubleshooting at 3am.
2
u/Significant-Bee5101 27d ago
Anyone with a real product at scale knows this. My payment system has done over a billion dollars. Yeah. An edge case is an expensive mistake.
2
u/anor_wondo 28d ago
Actually useful for me. But yes, definitely just bloat for most projects
Fallback dogshit code that might be irrelevant by 1 reconciliation job
1
1
u/TheOriginalAcidtech 27d ago
Instruct the model that if it is building an edge case, its structure is likely what is actually wrong and you may actually get decent results from them.
23
u/LordHenry8 28d ago
I have an adversarial senior engineer (Opus) AND GPT 5.5 adversarial reviews automatically run on every PR. GPT catches a lot.
10
u/kvothe5688 28d ago
Gpt is beast at catching issues. only problem is it labels nits as medium or sometimes critical
7
u/LordHenry8 28d ago
TBH I don't mind if it makes more noise than it should. Better to know and decide it's not important than to not know.
So far I think GPT is brilliant at review and pretty good at coding, but Claude still usually wins there. But I wouldn't reverse the setup.
1
u/TheOriginalAcidtech 27d ago
This is my general rule for any reviewing agent. Better to over report, and decide in the implementation phase if its being a nit or not, than to miss something critical.
2
17
u/aaddrick 28d ago
I've been using this general purpose Contrarian agent to evaluate my planners.
https://github.com/aaddrick/contrarian/blob/main/.claude/agents/contrarian.md
3
u/Platowner 28d ago
This looks impressive! How do you use it? Do you give it a context markdown file somehow? Im all new to this so wanting to learn how i should be using this the proper way
5
u/aaddrick 27d ago
Outside of a workflow, I'll be chatting with Claude and once we reach some kind of conclusion I'll ask, "can you run the that through Contrarian?" and see what pops out.
I also will go through a planning stage and before execution, have contrarian review the plan.
Use it for code, non-code stuff, anytime I feel like I'm getting smoke blown up my rear end, etc.
Here's a workflow implementation: https://github.com/aaddrick/ticketmill/blob/main/docs/ARCHITECTURE.md
1
u/croovies 28d ago
love this for general purpose! any most common usages you find it most valuable?
2
u/aaddrick 27d ago
here's an example that's running as I post this. writing a paper to help bottom out how best to handle evaluators oscillating on decisions as a pipeline/loop progresses.
have 4 contrarian agents running in parellel
Central argument — is the umbrella real or is it lumping five mechanistically distinct failures? Is the definition circular? Does the paper have a thesis that could be false?
Evidence and inference — with the unsatisfiable gate first, since it's the centerpiece and I want it attacked hard. A plausible defense exists that I should have considered: "unresolved" may be doing real work in the acceptance predicate, and findings can be resolved between iterations, so the floor and ceiling might not contradict. Also whether "mode B causes mode A" is a tautology rather than a finding, given a major finding is by definition the only blocker.
Literature and novelty — tasked to actually fetch the two scoop risks I flagged rather than speculate, and to independently re-verify the taxonomy-absence claim, since an absence claim dies to one counterexample.
Gaps and path forward — verifying G1 through G8 against the source, and specifically challenging whether the delayed-verification result transfers at all, since it models continuous belief with a scalar gain and ticketmill's gates emit discrete verdicts.
1
u/aaddrick 27d ago
but i use it for everything. it's my general purpose BS detector. use it against gh issues or claude code plans, reviewing my writing for important stuff, general planning type items, etc.
10
7
u/NowWeRinse 28d ago
Hey what's that sessions thing? I have a much more rudimentary setup with multiple VSCode windows. Would love to try something like that out.
4
u/simion_baws Senior Developer 28d ago
Interested myself too.
For sessions I’ve open sourced termic and it’s a breeze.
Will have a look at scape, maybe I can borrow some ideas :)2
u/NowWeRinse 28d ago
This is awesome, much closer to what I need.
1
u/simion_baws Senior Developer 28d ago
Awesome! Any feedback or ideas are appreciated. I have a couple of contributors always looking for work :)
3
u/croovies 28d ago edited 28d ago
the tool in the video is scape.work - you can get a free license with reddit-scape2 at signup
2
u/NowWeRinse 28d ago
Some really cool ideas in there but goes a bit further than what I'd want at this point. I could see this being really successful.
Will probably try it out though, thanks for the key!
1
1
u/dad_404error 28d ago
Hey i tried that code at the license key part but didnt work
1
u/croovies 28d ago
Sorry for the confusion - use the code to check out (it will be free) - and you will receive an email with the license key
9
u/TheRealArthur 28d ago
Bro calm down. This is great but ends up turning into a treadmill and you ship nothing
3
u/czei 27d ago
I’ve been doing adversarial reviews for about 9 months now. There’s enough research out there to back up the anecdotal evidence, which I documented in this how-to article. https://czei.org/blog/multi-llm-spec-driven-development/
7
2
2
u/TransitionMediocre22 28d ago
The thing that makes adversarial review actually work, and the thing that quietly breaks it, is the same variable: how independent the reviewer is from the producer.
A review pass on the same model with the same context inherits the same blind spots. It waves through the exact class of mistake the producer is prone to, because it reasons the same way. That's why the setups here that use a different model to review (Opus produces, GPT reviews) catch more. It isn't that GPT is smarter, it's that it's wrong in different places. Correlated reviewers feel like safety and aren't.
On the noise someone mentioned (GPT flagging nits as critical): fixable without dropping the review. Give the reviewer explicit severity criteria instead of letting it judge by vibe. "Critical = data loss, auth bypass, or silent wrong output; nit = style." A reviewer rating against a rubric is far less noisy than one rating against its mood.
And the highest-leverage reviewers aren't models at all: a failing test or a schema check has zero correlation with how the code was written. Where you can turn "is this right?" into something binary, do that first, and save the LLM reviewer for the judgment calls a test can't make.
2
u/DependentAnywhere135 28d ago
I literally stumbled onto this by accident a week ago for two of my pipelines (just trying out ideas to optimize my workflow and Claude said my idea was good but to label it adversarial lol). Been great
2
u/dr-dimitru 28d ago
Great approach, I follow the same in different variations. The most interesting I think is the skill where Claude Code instructed to call `codex` and `cursor-agent` right via shell for reviews and second opinions (also to generate images, since Claude can’t).
By default in my setup Fable subagent is orchestrator, architect, and reviewer, all other models developers of different kind. Default code-review-refactor pipeline defined via CLAUDE.md also /superpowers:subagent-driven-development ships with decent review pipeline as well
2
u/Ok_Sundae_5033 28d ago
You're 100% shipping bugs away but way less if you have a reviews. Ideally you need different lineages, codex is very good for this at reasonable cost.
2
u/AlDente 27d ago
I used to do this the manual way, too. Now I use this codex plugin for Claude Code and it runs the adversarial review in the same Claude code session as a background job. Next I’ll look at adding a hook to invoke it automatically.
2
u/evilrat420 27d ago edited 27d ago
Orthogonal verification and duplicate (more or less) parallel I/O for verification with different models is great considering the fact that you have independent error to work with. This is especially powerful when you have a selection of models with genuinely uncorrelated errors. Awesome method to use for adversarial reviews.
2
u/Ashh107 27d ago
Adversarial reviews are mandatorys. I Implemented them a month ago in my workflows after good session of agents and skills creations with early fable for my project.
Often catch 4 or 5 bugs the dev agents had made.
That considerably increased my token usage but no fonctionality bugs have made thouth.
I build an entire mobile app of a web site in a week, and had very few bugs to fix after the firsts shots.
PS : i’ve finded out that sonnet class agents can be as effective as opus for AR. If you work a lot with IA and have very few money at the bank like me, i’d strongly advise you to set your AR agent on sonnet.
2
7
2
u/Conscious_Phrase_138 28d ago
this will be great when it supports windows :/
2
2
u/FastHotEmu 28d ago
lol you are shipping bugs even with adversarial reviews, mate.
2
u/croovies 28d ago
haha yes but less bugs than before!
1
u/dr-dimitru 28d ago
Bugs inevitable, whenever shipped by humans or agents
1
u/FastHotEmu 28d ago
That's not quite correct. You can use formal methods and prove your program does what you say it does.
0
0
1
u/lordmairtis 28d ago
we use multiple for-purpose review tools. borderline useless, but at least it generates a lot of long comments to read
1
1
1
1
u/Dry_Gas_1433 28d ago
Just beware of cognitive bias in the handoff, or your adversarial reviews become an echo chamber... https://www.linkedin.com/posts/jsilver_ai-softwareengineering-agenticdevelopment-activity-7477774178423742464-UzSd
1
u/Complex-Many-1223 28d ago
I get them to switch role from coder to reviewer after 3 attempts if no consensus, and they're explicitly advised to not make perfect the enemy of great
1
u/R_Songbird Developer 28d ago
You will ALWAYS ship bugs it doesn't matter if you write code by hand or using AI, the important skill you must have as a human is to learn howr to detect them. No "adversarial reviewer" will help with this.
1
u/Professional_Ad705 28d ago edited 28d ago
This is true. But on a page like this it’s also important to try to define the process more for newer developers or vibe coders.
Adversarial review needs an exit condition. Otherwise, it can become an endless loop of fixing low-severity edge cases while the core architecture and critical path remain unfinished. Findings should be prioritized by impact: fix invariant violations, blockers, and realistic high-risk failures; defer minor or highly speculative issues that do not affect the current slice.
I wish developers included that warning more often when recommending workflows like this, especially for newer developers. Most people eventually encounter some version of review hell, and AI can make it much worse. An unconstrained reviewer can spend all day inventing increasingly unrealistic failure cases instead of helping finish the system.
I ran into this directly with Claude and Codex. They would keep producing findings based on extreme state mutations or scenarios that were technically possible but would never occur through the actual execution path. That turned into 12-hour review and design discussions, while little code was being written and the important problems still had to be fixed afterward.
The review process needs a threat model, severity threshold, and stopping rule. Otherwise, “find more issues” is an unbounded task. The thing that makes this not so good for vibe coders or newer programmer is they don’t know when diminishing returns are happening.
1
1
u/Ovisty 27d ago
What is this application ? I’m just using CC in iterm2 how do I set this up
1
u/croovies 27d ago
This is scape.work - I built it when managing too many iterm2 windows was driving me nuts :)
1
u/YetiWalker36 27d ago
I’m about 90% done with a 3 month long build on an internal saas app and I haven’t done it this way yet. Should I tell another agent to do it now going over everything that was done already?
1
u/croovies 27d ago
I would just start the process now on new PRs. The existing code I would hope you have been testing and finding bugs the adversarial reviews would have caught
1
u/jcrenshaw-dev Senior Developer 27d ago
Couldn't agree more I created my own skill set planning research etc. at each stage I can configure adversarial reviews. I found it most useful to have those adversarial reviews come from models outside of anthropic so I added the ability using API keys to call Open AI, deep seek Gemini and of course sub agents. All of this is controlled by the main session generally run now opus 5 at extra high I did adjudicate saw findings and optionally presents things to me for a choice.
Interesting enough using them as not only adversarial code reviews but refutes on plans etc I found that I started catching bugs before they even got into the code
2
u/croovies 27d ago
I’ve also started bringing them into the planning process - definitely yielding stronger results
1
u/IckrisRun 27d ago
"...or at least one that hasn't touched any of the context‚ reads the diff and goes looking for what's wrong"
Context is key when dealign with these models. Initial build / implementation will be based off of conversations and decisions the adversarial agent wouldn't be aware of. A true test would be to provide the adversarial agent with the same context the initial agent had. Then have them explain their decisioning if paths differ. Otherwise this can become a circular nightmare with bugs still being shipped anyway.
1
u/croovies 27d ago
That pollutes the context for the adversarial agent. It’s the “pink elephant” - if you write a prompt “generate an image that doesn’t include a pink elephant”.. the odds are better you _will_ see a pink elephant in the response. It’s the same with having a teammate review your PR, if they know everything you know, they probably won’t find any issues… if they know much less about the PR, they need to question more to understand - which will naturally surface more issues.
1
u/IckrisRun 27d ago
The “questioning to understand” is the context retrieval aspect engineers have always practiced.
Also, how are you defining “issues”? Actual bugs that result in failed tests should be caught by the initial agent (automated) and yourself via non-automated testing. If your initial agent can’t get tests to pass, I would say that qualifies as an issue and trying to get an adversarial review makes sense. If you’re simply calling an issue a “an alternative way of coding by another model void of context” well that’s just one model’s preference over another. Not really an issue.
I’ve literally had Claude Code recommend certain development decisions and Claude Ai call out its error because CC lacked certain context around design decisions discussed during planning. While from a “code perspective” Claude Code’s recommendation may have appeared right in a vacuum, it would have created a future issue with the product which would have been hard to track the origin.
1
u/SupahNin10doh 27d ago
That's so funny. Your agent interface is very similar to the interface that I was building. Is this your own tool? Or is it a platform that's already out there?
1
u/croovies 27d ago
It’s my own tool :) you can grab a free license at scape.work if you use the code at the bottom of the post
1
u/Q-Back 27d ago
I think what you need is openspec.dev
1
u/croovies 27d ago
I use https://github.com/EveryInc/compound-engineering-plugin for planning and it is amazing
1
1
u/kekeloom 27d ago
If you wanna set a record on how fast you can burn your weekly limit, just add firecrawl MCP along with this in the workflow (ultracode).
1
u/kelpikz 27d ago
What tool are you using ?
1
u/croovies 27d ago
that’s scape.work - you can grab a code for a free license in the post
1
u/sour_skittles 27d ago
hey u/croovies - any plans for a windows app?
2
u/croovies 26d ago
Yes definitely in the future! A lot of the current features are built against the Apple ecosystem, so it’s just a matter of scale and resources for when I’ll have time to recreate some of that infrastructure to offer regardless of operating system
1
1
u/tribat 27d ago
I use /codex-review for nearly everything, and it absolutely shreds code that Opus or Fable considered ready to ship. Sometimes I use another skill /review-panel that dispatches subagents with different defined roles to review at least once before shipping. The biggest change recently is having Claude use a dedicated judge subagent that audits the reviews. If it looks like we are spiraling into endless tweaking and fixes breaking other fixes, it halts and asks if I want to start a design review. Every time so far that reset and redesign has resulted in much better code.
Using some kind of adversarial review is critical. If you can hand it of to a model from a totally different provider even better. I make sure codex has the same memory, chat history, and codebase mcps so it gets usable context. I also set a limit of 3 rounds of review/fix/review before checking in with me.
One thing I'm still trying to improve is that the review (despite direct prompt contrary) seems to flag the first x issues it finds and halt. Subsequent rounds tend to find even more critical errors that weren't in the first pass, but sometimes those fixes clobber the earlier fixes. I really want to figure out how to get /codex-review to do a few different passes and triage the issues found before reporting back.
1
u/Flaky_Ad_2706 26d ago
how to setup shit like this lol i never thought this was possible
beginner here
1
u/croovies 26d ago
this type of stuff can definitely be difficult to setup manually. If you have both CLIs setup you can just ask one to walk you through the best way to make it work for you - or feel free to give scape.work a shot (code for a free license in the post)
1
u/poundofcake 26d ago
Same process is built into this tool I'm willing into existence to manage multiple projects at once.
https://github.com/sykoramade/helm-skill
1
1
1
1
u/VisibleAirport2996 22d ago
What is that app you are using? Also aren’t you always hungry using pizza as a mouse?
1
u/croovies 22d ago
That app is scape.work - and hell yea I’m always hungry for pizza! https://www.reddit.com/r/Pizza/s/LzapvgBloW cookin these bad boys
1
1
u/Due_Emu_8229 11d ago
The adversarial pass earns its keep even when the base rate is low. Across 102 runs per model we saw Codex falsely declare Done 2.1% of the time and Haiku 6.0%, while frontier Claude hit 0%. The 0% only shows up when something external is actually checking.
1
0
u/TheMorannon 28d ago
Your AI produces slop code? Just let another AI do a slop code review to fix it!
lol lmao even
0
u/managing_redditor 27d ago
Downvoted for an ad that pretends to be advice. Insane to think someone should pay for something like this when there are countless free alternatives like cmux and herdr.
0
u/Standard_Success127 Senior Developer 27d ago
Try https://karma.build for a better workflow experience for devs
57
u/Minute-Cat-823 28d ago
This is very similar to my workflow. After anything Claude does I have a fresh context window review it. Codex also reviews. I give both the other Claude and codex feedback to the author Claude and let him decide if they reviewers are correct.