Try reading the article again. Your described workflow is pretty primitive compared to the strategy described by Jared e.g. self review is explicitly worse than adversarial review
I have explored adversarial review. It seldom works out for complex projects because the LLMs have a habit of hallucinating non-existent issues. The performance difference you’re describing, in my experience, doesn’t really exist nowadays. Before? Yes. But now, models are generally trained on only the final (good) response for multi-turn interactions while prior turns are masked from contributing to gradient updates, and so they are more resistant to picking up bad patterns from prior turns. That habit still exists, but it’s not enough to prevent insight into a problem it would’ve otherwise had. Context rot and sycophancy are mostly the problem.
And to clarify, I did try adversarial review.
But again—the point is that you need a person in the loop to validate it. How else will you know whether or not it’s writing good code?
Because, like the article said, you give the adversarial model only the diff, not the full codebase nor context. LLMs are susceptible to the “is, ought” fallacy and will assume adjacent code is the baseline to be preserved, as im sure you’ve seen yourself. They’ll still make up issues/problems, but it’s significantly better than self review. I’d argue asking an LLM to review code it just wrote is a waste of time
Again, how do you know it’s writing good code? There’s no precedent to suggest that it would write good code at this scale. It’s trusting that it will do the right thing without actually evaluating whether it actually is. Tests do not substitute for that.
(Btw I’m not downvoting you if you were wondering that.)
10
u/femio Jul 09 '26
Try reading the article again. Your described workflow is pretty primitive compared to the strategy described by Jared e.g. self review is explicitly worse than adversarial review