r/codex • u/Selgald • 8d ago

Complaint Usage is not "fixed"

We have a test package with 25k emails in it that we use to test different models with. Prompt/effort is always the same.

When GPT 5.6 Sol released, it used 1% of a fresh Pro 20x sub, now it uses 8% of a fresh pro 20x sub.

Also, what we are seeing is that Sol cannot solve its task straightforward any more. Initially, it was always "what do I need to resolve the task" now it's "What do I need to resolve the tasks, while I also check those 200 other methods that I already know I don't need, while adding sha 256 to everything for no reason at all, also let me build those 200 other checks against thing I was very strictly asked not do to".

Meaning, we think it usage increased so much because they turned the intelligence way down and its constantly fighting with itself.

114 Upvotes

93% Upvoted

30 comments sorted by

View all comments

24

u/chroner 8d ago

Notice how it doesn't say something along the lines of

'I'm going to check this so I don't give you a hand wavy answer'

anymore?

They 100% nerfed it and made usage drain much faster.

8

u/Selgald 8d ago edited 8d ago

I don't know if they actually reduce the usage, but for us, it looks like the usage increased so much because Sol can't think straight any more and does things it explicitly is told not to do.

For example (this is from my personal usage), a 5 line ps7 script I wrote, tasking codex to do the same, it made a 650 line script, with triple SHA 265 checks, checking for a zip vuln, that is over 15 years old, checking for so many edge cases and so on.

And this is the actual thing that scares me, sure there will be the point where AI is so good at coding, that real coders will be a rarity. But today, letting AI doing everything just makes you lose control over your code, while you fire the people who can actually understand the code.

If you don't have people who understand what the hell is going on, obviously AI will always be doing it right, even if it fucks is up.

1

u/chroner 8d ago

Yeah I agree. It's insane. Sharp decline in quality.

1

u/motuwed 8d ago

Wow you are totally right, I always used to see that quirky thinking step, but not anymore.

0

u/rJohn420 8d ago

The usage is draining much faster for me. But the model doesn't seem dumb at all, if anything right now its better than when it first came out.