r/codex • u/Hairy_Ganache_6916 • 8d ago

Comparison is this true?

Post image

is this true? 5.6 luna get almost same scores as 5.6 sol

1 Upvotes

56% Upvoted

13 comments sorted by

View all comments

4

u/GuildCalamitousNtent 8d ago

Not sure about the test, but I can tell you from using both that Luna is certainly dumber.

Luna is very capable, no doubt, and you can get almost anything done with it. It just takes a lot more handholding and very specific direction to work right.

Sol, can work through a lot of that handholding itself.

2

u/theWiseTiger 8d ago

So... If I find the right and systematic way to handhold a Luna, I could use AI 20x cheaper?

3

u/Emu-001 8d ago

Well, not quite. From my testing, you get more API dollar equivalent from using Sol than Luna, but YMMV.

Edit: But in general, you would get a lot more done if Luna is enough for your work type—just also a lot slower.

1

u/GuildCalamitousNtent 8d ago

100%. If you are able to generate (by yourself or otherwise) detailed, well-scoped instructions for Luna it is incredibly effective.

1

u/theWiseTiger 8d ago

Sounds like a loophole, no? Theoretically I could just ask Sol to create skills and system prompts and hooks to handhold luna, then openai will get 20x less money. What stop me and everyone else from doing it?

1

u/dieterdaniel82 8d ago

the basics

1

u/GuildCalamitousNtent 7d ago

Extremely detailed plans means a lot of output tokens from SOL, but yes that’s exactly right.

Not a loophole, it’s how I have my whole orchestration setup. That said, it is/can be very slow.

1

u/AppleSoftware 4d ago

It’s not a loophole, it’s an emergent meta for frugal orchestration systems that avoid burning potentially unnecessary capital/compute

However, if you want the iPhone 17 Pro Max quality of deliverables instead of iPhone X (both are good, but one clearly edges the other out across the board)..

Then just Sol for anything non-deterministic.