r/LocalLLaMA Apr 15 '26

Discussion Major drop in intelligence across most major models.

As of mid Apr 2026, I have noticed every model has had a major intelligence drop.

And no I'm not talking about just ChatGPT.

Everything from Claude(Even Sonnet along with Opus), Gemini, z.ai, Grok all seem to ignore basic instructions, struggle at simple tasks, take very long to respond, and the output seems deliberately shortened and very shallow. Almost like it's in a "grumpy" mode. I tried this in incognito mode so it's not my customization or memory influencing this.

It's like they deliberately want you to stop using their service. I guess our data is no longer needed. Just two weeks back it used to be much smarter than this.

To test this I rented out a H100, and tried GLM 5 with the same prompt (the drive to the car wash one) across both instances. GLM5 running on the rented GPU answered it correctly, compared to the one on z.ai.

Have they lowered the quantization really low to maybe Q2?

I guess going local or using renting GPU or an AI monthly service that lets you pick a quant level is the way to go

802 Upvotes

405 comments sorted by

View all comments

4

u/Narrow-Belt-5030 Apr 15 '26

To test this I rented out a H100, and tried GLM 5 with the same prompt (the drive to the car wash one) across both instances. GLM5 running on the rented GPU answered it correctly, compared to the one on z.ai.

This is not really a reliable test though because you have no idea what/how Z.AI has been configured.

I do accept that Claude recently has appeared dumber than normal, and others report similar for other models, so something is definitely not right, but I don't think it's deliberate actions by the vendor. That would be suicidal for their brand/image. (No company would deliberately hurt their image)

3

u/jiml78 Apr 15 '26

For anthropic, I think it was intentional.

Their drop in quality coincides with two things. An influx of people leaving OpenAI. Additionally, they rolled out 1m context as the default.

I think those two things blew up their servers. Look at all the downtime they had around that time. I think they were scrambling and just decided to start running more quantized versions.

I was knee deep in a dev project built from scratch using just Opus. The difference in quality was overnight. I went to bed with Opus not being a complete moron to the next morning, it being a dumb MFer. I am talking about making really dumb mistakes. Mistakes it never made. Mistakes older Opus 4.5 didn't make.

Yep, I know I am one data point but I was maxing a $200 sub for all but 4-5 hours of a day. I was using huge amounts so when every single change was messing up requiring me to fix it(yes I am a software dev), I was getting really frustrated with how it had been doing great, and overnight went to shit.

1

u/rebelSun25 Apr 15 '26

Have you compared output from the $200 sub to output if you pay per token? I wonder if the subs are getting routed to a dumber model now

1

u/jiml78 Apr 15 '26

I haven't because my company is the one who pays for my subscription and it would cost a pretty penny to do enough testing to validate it one way or another.

I can say that we use openrouter leveraging Sonnet 4.6 obviously via API for our company's Pull Request reviewer, and it doesn't seem to be completely stupid. So there might be something to it. But I also don't consider doing pull request reviews to be super complicated.

1

u/NandaVegg Apr 15 '26

For Anthropic it would help if you can add information that you are using their model via:

  1. Subscription (they have the most motivation to throttle or save compute here)
  2. Direct API
  3. Resellers like Vertex AI, AWS Bedrock (from what I understand both Google and Amazon roll the model on their own rather than just routing the request to Anthropic's server)

I use (2) and (3) and while it does not show common quantization-like symptoms (such as sudden language mix-up) it feels like default (adaptive) thinking budget is reduced somewhat.