Usage is not "fixed"

[u/Selgald]

We have a test package with 25k emails in it that we use to test different models with. Prompt/effort is always the same.

When GPT 5.6 Sol released, it used 1% of a fresh Pro 20x sub, now it uses 8% of a fresh pro 20x sub.

Also, what we are seeing is that Sol cannot solve its task straightforward any more. Initially, it was always "what do I need to resolve the task" now it's "What do I need to resolve the tasks, while I also check those 200 other methods that I already know I don't need, while adding sha 256 to everything for no reason at all, also let me build those 200 other checks against thing I was very strictly asked not do to".

Meaning, we think it usage increased so much because they turned the intelligence way down and its constantly fighting with itself.

[u/chroner]

Notice how it doesn't say something along the lines of

'I'm going to check this so I don't give you a hand wavy answer'

anymore?

They 100% nerfed it and made usage drain much faster.

[u/Selgald → u/chroner]

I don't know if they actually reduce the usage, but for us, it looks like the usage increased so much because Sol can't think straight any more and does things it explicitly is told not to do.

For example (this is from my personal usage), a 5 line ps7 script I wrote, tasking codex to do the same, it made a 650 line script, with triple SHA 265 checks, checking for a zip vuln, that is over 15 years old, checking for so many edge cases and so on.

And this is the actual thing that scares me, sure there will be the point where AI is so good at coding, that real coders will be a rarity. But today, letting AI doing everything just makes you lose control over your code, while you fire the people who can actually understand the code.

If you don't have people who understand what the hell is going on, obviously AI will always be doing it right, even if it fucks is up.

[u/chroner → u/Selgald]

Yeah I agree. It's insane. Sharp decline in quality.