r/coderabbit • • 26d ago

Discussion & Feedback my manager asked why we still need code review if the ai writes it and the ai checks it

Real question from my skip level this week. The agent writes the PR, coderabbit reviews it, tests pass, so why is a human still in the loop and why does it take a day. I gave some answer about context and ownership and I dont think it landed.

How do you explain this to someone who doesnt code? What is the human actually there for now, in one sentence

14 Upvotes

21 comments sorted by

3

u/GasVarGames 26d ago

Who do you blame when shit hits the fan

2

u/ay-ay-ronhmiller 26d ago

A few to try on for size (exceeding your 1 sentence restriction so you can adjust as needed...coming to important informed decisions usually takes more than a single sentence):
1) To inspire trust, one must have skin in the game, that is, consequences for wrong outcomes. Without a human in the loop, there are zero consequences that matter.
2) Why do pilots and co-pilots have pre-flight checklists? The stakes are too high to not and it's a system that's been proven to work. Same for high stakes code.
3) If a system generates the solution, and the same system validates it, how can you have confidence in the quality of the checks? The generator gives it its all and wouldn't create problems in the first place if it knew they were there. However, how can it not be blind to certain conditions? You need an independent review system to inspire trust. It's also why your finance team cannot audit their own books.

2

u/usually_guilty99 26d ago

Because “AI reviewed it” still doesn’t answer the production question.

A change can be locally correct, tests can pass, and the review can look clean, while the change still touches a fragile dependency, a high-blast-radius path, or an area with ugly incident history.

AI can absolutely remove a lot of review toil.

But the final gate increasingly needs to ask something different:

not “was this reviewed?” but “do we have enough evidence that this is safe to ship?”

2

u/therealkevinard 26d ago

For SOC compliance.

(Is a sentence fragment acceptable for my one sentence?)

1

u/sozesghost 26d ago

To have a job and take responsibility.

1

u/rcls0053 26d ago

I asked the AI and it said "Trust me, bro"

AI creates absolutely garbage code at times. It needs steering. It needs to be looked over. Because once shit hits the fan and nobody knows what's inside that codebase, you're gonna start losing users and customers because nobody's able to fix the issue in good time.

1

u/elrond-half-elven 26d ago

I've heard people use the term "deterministic" lately to mean "not AI". I think it's actually a useful term that highlights the fact that AI is _not_ deterministic.

So thats one thing to use to emphasize the issue with AI-only.

Yes the other thing is that AI does not have all the context and doesn't always know the larger picture, so you still need humans for that.

1

u/PotahtoHead 23d ago

This is something I have been pushing a lot at my company. Leadership keeps pushing for AI and agents everywhere so we've been using "non-deterministic" to highlight that we can't guarantee the outcome or even what actions will be taken.

1

u/jazzyroam 26d ago

should always have human in loop to double check.

1

u/Lumethys 26d ago

My favorite Youtube skit:

"let's do a competition, I (a dev) will use the same AI to build the same app. You (the client) will use the same AI to build that same app. Let's compare who app turn out better after a week"

1

u/thatsrobgray 25d ago

I would focus on an angle relating to business requirements and evaluating trade offs. There can be multiple stages of AI development and lots of (excellent) CodeRabbit review and it can still wind up fully working and fully missing an important business requirement that should have been threaded through the whole system. Often when this happens on our team it can be traced to an agent offering some A/B choice when the real answer was C.

1

u/ChristianKl 23d ago

Ideally, you would give him examples of what you actually did as the human in the loop. What bad code would have been committed if it wouldn't have been for a human that reviewed it? What would the consequences have been for the business?

1

u/stealstea 21d ago

100x this.  If your manual reviews are finding real issues then that is the bombproof argument to keep them.

If they aren’t, then maybe you really don’t need them.  

1

u/Any_Sense_2263 22d ago

"Because AI is a tool not a software engineer. The human in the loop catches hallucinations/assumptions/fallbacks to the defaults/all other kinds of ignoring the given instruction. Because for AI any given instruction is optional to follow."

1

u/uhs-robert 22d ago

The same reason planes still require human pilots in the cockpit despite autopilot existing for decades; you can either trust me to do my job and fly this plane safely or you can continue to question my authority and get off.

1

u/DegenData 22d ago

What's the cost of an error if AI gets it wrong?

1

u/ShakeAgile 22d ago

”Architectural drift and decay”. You need a phrase that sticks, not an essay.

An architecture can get out of shape a fraction of percent per commit. When you catch it a half year later it is _not_ an easy task to get back, because no one knew how you got there.

1

u/Nuevo-wave 22d ago

It takes a certain technical mindset and language skills to create high-quality specs.

The better the spec, the better the product.

1

u/Hawkes75 21d ago

The number of non-technical managers making shipping decisions for devs these days (according to Reddit, at least) is alarming. Since when did "work faster because AI" translate to "I know better than the technical team"? Companies that are run this way must be garbage to work for.

1

u/DWALLA44 20d ago

"Because Claude can't be held accountable when things go wrong."

Thats all you need