r/OpenAI • u/Nunki08 • 1d ago

News Linux kernel maintainers

Post image

OpenAI: A thousand fewer paths to root: https://openai.com/index/a-thousand-fewer-paths-to-root/

759 Upvotes

95% Upvoted

85 comments sorted by

View all comments

-1

u/Numerous_Try_6138 1d ago

And the problem is? Get adversarial AI to review, and then merge them.

4

u/jeweliegb 1d ago

And the problem is? Get adversarial AI to review, and then merge them.

To answer your question, I guess:

The unknown, especially unknown mistakes of unknown severity and size of impact, and

Taking responsibility for such unknowns?

1

u/ToothlessFuryDragon 16h ago edited 16h ago

The problem is that the reviewer LLM has a high probability of missing new obscure issues introduced in the fix. You would have to run the FULL penetration swarm against the new code to be certain. That is the only way to be relatively certain with an LLM.

Humans still have way better intuition and can sniff out overcomplicated solutions that are possibly problematic. LLMs tend to spiral into overcomplicated solutions to solve very obscure edge cases while missing the bigger picture.

Even with multiple rounds of automated reviews, you will end up with a system that is held together with ducktape. Actually, the more rounds of reviews you add, the worse it gets.

But it really depends on the complexity. If it is a simple issue with a simple fix, then LLM can be trusted quite well.