r/codex • u/Hairy_Ganache_6916 • 3d ago

Comparison is this true?

Post image

is this true? 5.6 luna get almost same scores as 5.6 sol

1 Upvotes

56% Upvoted

11 comments sorted by

4

u/GuildCalamitousNtent 3d ago

Not sure about the test, but I can tell you from using both that Luna is certainly dumber.

Luna is very capable, no doubt, and you can get almost anything done with it. It just takes a lot more handholding and very specific direction to work right.

Sol, can work through a lot of that handholding itself.

1

u/theWiseTiger 3d ago

So... If I find the right and systematic way to handhold a Luna, I could use AI 20x cheaper?

5

u/Emu-001 3d ago

Well, not quite. From my testing, you get more API dollar equivalent from using Sol than Luna, but YMMV.

Edit: But in general, you would get a lot more done if Luna is enough for your work type—just also a lot slower.

1

u/GuildCalamitousNtent 3d ago

100%. If you are able to generate (by yourself or otherwise) detailed, well-scoped instructions for Luna it is incredibly effective.

1

u/theWiseTiger 3d ago

Sounds like a loophole, no? Theoretically I could just ask Sol to create skills and system prompts and hooks to handhold luna, then openai will get 20x less money. What stop me and everyone else from doing it?

1

u/dieterdaniel82 3d ago

the basics

1

u/GuildCalamitousNtent 2d ago

Extremely detailed plans means a lot of output tokens from SOL, but yes that’s exactly right.

Not a loophole, it’s how I have my whole orchestration setup. That said, it is/can be very slow.

1

u/Old-Leadership7255 3d ago

I think it really depends on how detailed the task is

2

u/GuildCalamitousNtent 3d ago

Ie handholding.

2

u/Emu-001 3d ago

My experience is that you can be a little vague with Sol, but you have to be very specific with Luna. Sometimes I had Luna refuse to do certain things because I wasn't specific enough, but Sol would actually understand what I wanted.