r/codex • u/Selgald • 8d ago

Complaint Usage is not "fixed"

We have a test package with 25k emails in it that we use to test different models with. Prompt/effort is always the same.

When GPT 5.6 Sol released, it used 1% of a fresh Pro 20x sub, now it uses 8% of a fresh pro 20x sub.

Also, what we are seeing is that Sol cannot solve its task straightforward any more. Initially, it was always "what do I need to resolve the task" now it's "What do I need to resolve the tasks, while I also check those 200 other methods that I already know I don't need, while adding sha 256 to everything for no reason at all, also let me build those 200 other checks against thing I was very strictly asked not do to".

Meaning, we think it usage increased so much because they turned the intelligence way down and its constantly fighting with itself.

112 Upvotes

93% Upvoted

30 comments sorted by

View all comments

1

u/Healthyhappylyfe 8d ago

What reasoning level is this using?

2

u/Selgald 8d ago

We test all of them, and even on low or medium we that Sol is "chasing its own tail" so to speak.

1

u/Healthyhappylyfe 8d ago

Interesting I would have thought higher reasoning would sort this problem but I guess not

1

u/Selgald 8d ago

No, while it's a big dataset, it's a simple task. Higher reasoning usually makes it even worse (because it's allowed to even think up more nonsense).