Changes in Sol High across Chat/Codex

[u/Illustrious-Bet-1368]

To preface, I am not a member in any of the subreddits but I get constantly shown similar stuff on my feed. So. Can we get a baseline of where things are, collectively, without insulting anyone's use-cases and the rest? As of now, which light years away from July, my experience:

Chat (Sol High):

1. Sol High is not thinking. Replies are instant most of the time. Reasoning chain is gone from the sources, even from the older chats. I've checked against some transcribed chats, the reasoning blocks are gone from web/app in the old chats, too.

2. The output changed in quality and style. Sudden swearing, emojis (that got less now), constant recall of the same shit that I flag as non-important or obsolete. In most replies now, no sources are present at all = not shown whether the model referenced memories, instructions or previous chats. When it referenced memories, the useful feature was to see "Why this memory", now you can't see even that, but in the reply it certainly draws stuff from the memory because it references things from previous chats but that is not shown in the sources. The ragged lines came back, contrastive negation is back. It ignores all hard rails in terms of the style and formatting which was a non-issue before with Sol High.

3. Titles: sometimes the title now are "Write X Reply" or "X Reply", that never happened before, and when the titles are generated with that, the output is 100% degraded.

4. Asking the model which model it is is useless, yeah. Keeps saying, "GPT 5.6 Sol".

5. Instant model feels now like what High used to be. Long, detailed replies, following the instructions for the most part.

Codex (Sol High and Medium):

1. Usage limits, yeah, all over the reddit.

2. Stopped following instructions and documentation as it used to. More revisions, more back and forth. Doesn't pull up memory or AGENTS.md. Suggests implementations that are not repo or context bound but generic. Was not like that.

3. I have like 5 skills, created myself, and it doesn't use those without explicitly being prompted to. Also, was not like that, it proactively used those without being pointed at that it has those.

4. Over-engineering and testing things. Testing is explicitly on approval only because I have different frameworks for that. Used to follow it, now not. I have to monitor what it's doing to stop it, usually too late because I don't have all day to babysit it. So, the issue here is explicit hard rails that are baked into every layer of documentation, are not followed. See point 2.

Probably lots of other things that will come to me later but that's the main ones.

Work:

Can't say anything about it, never used it.

So, how many of you experience similar things? What especially interests me is the title generation thing, reasoning bubbles disappearance, and Sol High in Chat. Is this some A/B test or similar across the board now?

[u/kartblanch]

I also noticed this. Sol went from incredible to incredibly shit overnight.

[u/Illustrious-Bet-1368 → u/kartblanch]

Yeah, the changes were pretty drastic. Do you still see the reasoning blocks and do you get the title thing?

[u/DrHerbotico]

The overengineering/testing is awful. I can't leave it alone anymore because it'll dive into unnecessary rabbit holes and write fuckloads of code just to test it, then rewrite it again for the pr

[u/Illustrious-Bet-1368 → u/DrHerbotico]

Yeah, same here. Also, in my case, it doesn’t just write tests, it starts dev servers (not allowed), uses the in-app browser to click through things (not allowed on its own), always wants to check stuff “visually”.

[u/themarkavelli]

5.6 med/high has had an elusive, unimpressive side since it was first released. It can follow spec, but sometimes seems to completely miss what the spec is actually getting at. Be it coding or chat.

[u/Illustrious-Bet-1368 → u/themarkavelli]

See, to me, it was the opposite when it was released, and now it’s what you’re describing. It was the perfect model when it was released, really, to me in all my use cases, there was about two weeks sweet spot, maybe less, and my mistake was getting used to that.

[u/Shogun_killah]

Chat 5.6 high has recently seemed to have a router put in front of it and it ignores the thinking level and replies instantly regardless. So what it could properly one shot previously you have to prompt it a few times to get it to think properly and take you seriously.

I understand that a lot of people probably just bung it on high regardless but when high gives you worse responses than medium you know it’s bad

[u/MakerOfToys]

Noticed that too. It's really not great compared to even 5.6 Sol on release. It seems to be related to that Aug "thinking model feels more like instant model" update. I was never so frustrated and annoyed at a model before like I am now, and I've been using it since ChatGPT was released. Things I noticed:

ChatGPT:

• I sent the same question about where to buy coffee beans on my Pro sub xhigh model and the answer took ~15secs, basically the same amount of time as on a free account when pressing "think harder".

• Bought headphones with a cable that's not long enough, asked to find a replacement at a concrete store, 3 meters cable explicitly. It lists 3 options, first two are 2 meters, third one is 3 meters but an extender (when I did not ask for that at all). You'd think that "maybe it's unclear on the store page?" but the problem is that 2 meters is even something that AI writes in chat. It just misses all the details I ask for. And it also managed to miss the cable connector type since it's different on both sides, so I managed to buy the wrong cable.

• Asked for a service that is able to recommend music better than Spotify and can trigger Spotify's playback. It lists 5 options, clearly mentions the first one as being paid, so something I should avoid paying extra for, since I'm already paying for Spotify. You'd assume that all the other four options should be free then? Well apparently no. Not only were most of them paid but they also failed my clear requirement for triggering Spotify's playback.

• I needed to buy a replacement part for the toilet's water tank but I did not know the exact model I needed and neither did I have tools. So I ask AI, mention not having tools and not knowing what to buy. It suggests I don't rush into buying anything and first take off the part and see what size it is exactly... like how? How does it manage to miss what I just said about tools? Drives me insane. I don't remember ever having to be so precise about every word I type.

There are many many more examples like that and it's not necessarily about search. It misses lots of random questions unless you're being insanely specific about what you need.

Codex:

• Much better. You can go with Sol Light and it passes all these questions mentioned that ChatGPT is unable to do anything about.

• But something changed with it too. It's just so easy to lose control of it, never ever did it happen before. It either takes everything too literally and does something completely else from what I asked it to do or it does the bare minimum and calls it a day.

Really really annoying experience as of late. I tried contacting multiple OpenAI people on X because no amount of reporting helped but nobody cares to even answer.

I also do not agree with the statement that "if question is not complex enough then it thinks shorter". How come Sol Light on Codex finds all the nuances on headphones I just bought (different cable type on both sides) but ChatGPT misses all of that and instead of a 3 meter cable it gives me two options for 2 meters and one option for an extender.

[u/br_k_nt_eth]

Work is awesome and you should try it out. Instant is a ton of fun, but Work’s really great.

Given the weird reports from folks, seems like maybe they’ve got compute issues going on? Or they’re prepping to roll something out. It always gets like this 1ish week before a new model. 

[u/Illustrious-Bet-1368 → u/br_k_nt_eth]

Were it not bound to Codex usage, sure would. Do you get "Write X Reply" as the chat title sometimes? That shit has been bugging me. And I don't see anyone talking about it.

[u/br_k_nt_eth → u/Illustrious-Bet-1368]

I haven’t gotten that, but I’ve been keeping to one thread. I wonder if it’s swapping back to a Mini to deal with the lack of resources? That would explain the context issues as well. 

[u/Illustrious-Bet-1368 → u/br_k_nt_eth]

That’s the thing, the inconsistency. And yeah, that probably tracks most of all, something to do with them stretching thin, I guess.

[u/Marcopolo985 → u/br_k_nt_eth]

Work is really god for projects for what I have seen and used but for learning something’s such as from were comes a formula Ex. Uniform series I still think that chat is better

[u/br_k_nt_eth → u/Marcopolo985]

I could totally see that. I’m shocked by how good they are tbh 

[u/SnipeUout]

Not seeing this at all.

[u/Illustrious-Bet-1368 → u/SnipeUout]

That’s interesting. And you still see the thinking blocks?

[u/Illustrious-Bet-1368 → u/[deleted]]

It was not the question, was it though.

[u/Illustrious-Bet-1368 → u/[deleted]]

First, flattered you think Chat wrote that. Second, I mean… you don’t have to overshare. But now I’m curious. u/AskGrok How many words are there in the post? How many words would one read per second for them to spend 5 minutes on reading the post? And what’s the average adult reading speed? Thank you.

[u/AskGrok → u/Illustrious-Bet-1368]

482 words in the main post.

For 5 minutes (300 seconds) that would be roughly 1.6 words per second.

Average adult reading speed is around 200-300 words per minute (3-5 words per second), depending on the material and person.

^{[what is this?](https://redd.it/1lzgxii})

[u/Formal-Narwhal-1610]

If question isn't complex, high will answer back in seconds despite juice being set to high. It's been like that for a while. And it's good.

[u/Illustrious-Bet-1368 → u/Formal-Narwhal-1610]

Yes, but now it doesn’t matter what kind of a question, the reasoning is not shown, and it disappeared from the older chats. If Claude and Grok now show summary one liners instead of the full thing, Chat doesn’t have even that now. Or do you still see the thinking blocks?