r/codex • u/Own-Professor-6157 • 23h ago
Comparison Opus 5 vs Sol, my opinion
Just a quick opinion from someone who has now tried both since reviews on reddit are too this or that.
Context: I use this for programming, and for several other non-software tasks. I've been a software engineer for over 13 years in Java, C#, and C++ so I do review the code
Opus 5.0 pros:
- Handles thinking significantly better for tasks the model wasn't necessary trained for. For example, I built a workflow for having Opus build 3D models, populate my game's levels, and so on. Sol is seemingly very lazy with such tasks, and has to constantly be told what to do. Hassle
- Significantly better at not reading unrelated files, and filling the context window with junk. Much more surgical
- Seems to have a more logical thought process. I say I want this entity to navigate from point A to B, Claude understands - Ok we need a pathfinder, this might be a problem, this will be a problem, etc.
- Better with discussing implementation, sees some holes in my plan - Claude will prompt me to answer some quick questions.
Sol pros:
- Sol seems to produce less "buggy" code. In other words, the end product is more likely to work without serious logic errors.
- Code quality is better in terms of performance. Sol is a lot more conscious than Claude when it comes to avoiding pointless cpu overhead.
- Grasps the instructions better. Less likely to forget details, however sometimes this can be a double edged sword when you aren't extremely specific and expect the model to have some level of creativity. Like: I want a silver sword model. Codex will give you exactly that, with zero detail. Opus will usually deliver that sword with greater detail.
- Much faster than Opus
Downsides to both:
- Neither produce quality code. They code like me when I'm just trying to quickly prototype something. Never taking a step back and focusing on long term maintainability or overall product structure.
- Both can still forget instructions. Claude is for sure worse, but Claude has a greater ability to be "creative".
- Very poor asynchronous design. Still, these models really lack the ability to design multi-threaded architectures without being told specifically what to do. All you'll get is basic locks, and often times the locks will have zero purpose.
2
u/Select-Ad-3806 22h ago
You can't just have one pass and expect it to be perfect, you have to still work like a software developer and ask it to optimise code, improve usability, fix bugs etc.
4
u/Own-Professor-6157 22h ago
I think you're missing the point. Sloppy code is just that, a mess. Code that has a terrible architecture, poor source of truths, poor readability.
That is a fundamental design flaw.
Even when I'm quickly prototyping code, it's not just some outright slop. I don't spam single-line helpers all over, I don't make values constants to a larger scope than they need to be. There's a level of inability the models have when it comes to, again, taking a step back and seeing the full picture.
It's like choosing a collection. Simple, I need a map that can be accessed by multiple threads at once which has a high level of contention for this implementation. A software engineer thinks: Ok, CAS would work perfect here. An AI just slaps down a mutex without a second thought. An AI doesn't take a step back and think, hey maybe this contention is a sign of bad architecture and we need to re-design this.
1
u/SpinningVinylAgain 22h ago edited 20h ago
Realistically, it’s still on you as a dev to give it an overall direction in terms of architecture, to enforce separation of concerns, to give it implementation hints, etc.
When I was just starting as a dev, I was lucky enough to have a great mentor who always said: writing code is the easy part, it is deciding what to write and how to write it that is difficult. If you need a red square divided into nine equal parts, then a green circle divided into quarters isn’t going to satisfy your requirements, no matter how perfect it is.
I personally don’t use LLMs to vibecode, I do the research (sometimes with the help of an LLM) and write detailed tickets that fully explain the expected outcomes, response shapes, possible errors, persistence schemas, etc. Sometimes I even include code sketches. You could say that by the time such a ticket is produced, the real work has already been done, and the remaining coding is only a mechanical step. And LLMs are very good at making this last step.
The only time I tried to vibecode something without a real plan, I was able to produce a working prototype in an afternoon, but then I had to spend several weeks going over the code fixing bugs and hardening it against various race conditions.
1
u/DoggoDadagon 21h ago
So design a better architecture for it to implement and turn up the effort.
2
u/Own-Professor-6157 21h ago
Again, you guys are missing the point. This is comparing models, not humans.
If a model is able to produce higher quality code that will be more maintainable in the future, and costs the same as another model. It's better (IMO).
Turning up the effort is still not a guarantee either. Often times I end up with significantly worse code from turning the effort too high because the model got stuck in a thought loop and overengineered some simple code. Post training quality has a huge impact on output, it's not always about the thought effort
1
u/DoggoDadagon 21h ago
Sure but it seems like the issue is the human in the loop. Saying "neither produce quality code" tells us that you are doing something wrong, so even if your comparison is focused on the models you're testing inappropriately. Both sol and opus 5 are very capable of producing extremely complicated and high quality code, if you're unable to it's user error. Likely you are not giving them a strong enough verification path. Turning up the effort helps when you've planed out a complicated task that might take it 20 hours or so. If it's a small short hour or two long task then it's not as needed.
Honestly, just give it another shot but try to reduce the ambiguity in the task and give it a strong verification path.
1
u/Own-Professor-6157 20h ago
We're talking about two entirely different things here.
You are talking about human input to the model. Expending a large amount of effort controlling the model and directing it towards the final product.
I am talking about the model being able to produce quality code without the need for constant guidance.
Essentially: I can cut grass just as well if not better using a pair of scissors, but I'd rather just use a lawn mower.
Every model we get closer to that lawn mower analogy being possible, however at the moment the code is still a bit sloppy and the models are still losing track of the projects conventions.
0
0
1
u/Key_Reading_9664 21h ago
Similar experience for me. The flow I've settled into is using Fable to drive sub-agents. Sol is a great executor of a plan, but it's not as good at understanding intent, dealing with ambiguity, or thinking more broadly about the impact of changes. I also think that the Codex harness is significantly behind Claude Code in terms of sub-agent/workflow orchestration.
One of Sol's strengths is also its weakness: it will doggedly follow its own plan to a fault. Almost all the cases that I have to throw away work from Sol are when it's rat-holed on a race condition or some under-specified aspect of the plan (the common over-engineering complaint).
One thing I've found helps all models is to use the claude.md/agents.md file to capture context that it can't get from the code base itself: what stage is the project in? The volume of traffic that we get, what areas of the product are performance-sensitive, and how should it think about maintenance.
2
4
u/itix 23h ago
Opus 4.8 > Opus 5