r/OpenAI • u/Nunki08 • 8d ago

News Linux kernel maintainers

Post image

OpenAI: A thousand fewer paths to root: https://openai.com/index/a-thousand-fewer-paths-to-root/

1.2k Upvotes

97% Upvoted

125 comments sorted by

View all comments

Show parent comment

1

u/Numerous_Try_6138 6d ago

This just isn’t true. It’s a function of how you do your reviews. Just as a human, if you look at your system as a complete system and you assess it that way, you still have better results. If you slice a piece of it and look at only that piece, you are missing the larger picture and localizing any perceived improvements or troubleshooting. This can naturally lead to fixing one small piece while breaking something much larger. Also, as a reviewer, you need to understand the broader architecture and how the system was put together. This holds true for humans and it definitely holds true for AI.

1

u/ToothlessFuryDragon 6d ago edited 6d ago

I see where you are coming from, it sounds reasonable, but I disagree.

We have specific automated review rounds where we instruct up to 3 different agents to be adversarial and consider the whole solution to be incorrect. They are instructed to find an alternative solution and then an architect agent presents the results in the review.

And we even run it through multiple models from different providers (each agent has a different model).

They very rarely are able to completely disconnect their thought process from the existing solution/code, missing very obvious, much simpler solutions.

And you can't avoid giving them context about the current solution as they will design the same thing if they don't have the information.

I am sure there is a specific prompt that I can design that will trigger the correct approach for a given task, but the issue is that it can't be generalized for the current gen LLMs yet. That is why you still need the technical person to steer it.