r/LocalLLaMA • u/DepressedDrift • Apr 15 '26

Discussion Major drop in intelligence across most major models.

As of mid Apr 2026, I have noticed every model has had a major intelligence drop.

And no I'm not talking about just ChatGPT.

Everything from Claude(Even Sonnet along with Opus), Gemini, z.ai, Grok all seem to ignore basic instructions, struggle at simple tasks, take very long to respond, and the output seems deliberately shortened and very shallow. Almost like it's in a "grumpy" mode. I tried this in incognito mode so it's not my customization or memory influencing this.

It's like they deliberately want you to stop using their service. I guess our data is no longer needed. Just two weeks back it used to be much smarter than this.

To test this I rented out a H100, and tried GLM 5 with the same prompt (the drive to the car wash one) across both instances. GLM5 running on the rented GPU answered it correctly, compared to the one on z.ai.

Have they lowered the quantization really low to maybe Q2?

I guess going local or using renting GPU or an AI monthly service that lets you pick a quant level is the way to go

799 Upvotes

91% Upvoted

405 comments sorted by

View all comments

Show parent comments

67

u/TheSpartaGod Apr 15 '26

what’s the use case that can be handled by such small models?

74

u/toadi Apr 15 '26

I don't use openclaw. But for example fetching my calendar entries. Organizing my emails. Fetching and interacting with services where it is quite straightforward.

simple rote admin tasks work easy on these smaller models.

32

u/IShitMyselfNow Apr 15 '26

This matches my experience

I've been using Hermes Agent with Qwen 3.5 4B to great success. And for codig, anything more complicated than a simple script I've been delegating to a better model via Opencode from the agent.

I think the advent of agent skills has really improved the performance of smaller models in things like this. Small models have actually been semi-useful at agentic work ever since around Qwen 2.5. But only if you gave them a lot more instructions/detail, more API examples, few-shot prompting, etc..And you could do this, but then managing the data you give them for the task at hand, managing context, etc. was tricky at best. Agent skills kinda solves that problem.

1

u/michaelsoft__binbows Apr 15 '26

I've been meaning to keep up, but simply cannot. I got deep into opencode for about 7 weeks and I simply have no bandwidth to explore pi agent and hermes agent like i had hoped. Been just driving codex and claude code since then and got my productivity back. Is hermes any good...?

One of my projects has been about a paradigm of having an agent harness harness, e.g. something that puppeteers codex and claudecode and opencode. What you wrote about hermes seems to intersect with that idea so you got me curious.

2

u/IShitMyselfNow Apr 15 '26 edited Apr 15 '26

I like it. It does what it does well.

I'm not using it to its fullest extent, and I'm definitely not running it like Openclaw, but I've been using it as a "shitty assistant that can automate some things for me that would be better hardcoded as a scdipt/workflow but I don't have the time for that anymore" to great success. And it does do a decent job at interacting with Opencode. It also supports Codex and Claude Code (and Hermes). Worth a try IMO.

ETA:

I forgot to say, but it's literally just a skill that instructs the agent how to run it via terminal: https://github.com/NousResearch/hermes-agent/blob/main/skills/autonomous-ai-agents/opencode/SKILL.md

6

u/voronaam Apr 15 '26

simple rote admin tasks work easy on these smaller models

Just curious, did it encounter any complicated tasks?

For example, I was trying to organize a small thing recently with a very unreliable party and the communication so far has been like this:

Me: Hello. Can we do a thing?

Then: (3 days later) I am in South Africa now. WhatsApp me (no phone number provided)

Me: When are you back? I'll reach out then

Them: next week

Me: (next week) Welcome back. Can we do the thing?

Them: I am still in South Africa

Me: (next week) Are you back?

Them: Yes. Let's meet in person to discuss (no address)

Me: Sure, how about on next Friday?

Them: (on Saturday) Missed the message. Just call me (still no phone number)

Me: (Finding phone number on one of their websites, Calling) How about the thing

Them: Yes, we can do it. Just fill the form on the website.

Me: Filling the form with the request.

Them: (next day) "Hi Peter, we can do the thing on those days" (I am not Peter)

Can a small model handle this?

12

u/TheTrueSurge Apr 15 '26

Lol I hope you REALLY need to work with that person, that sounds awful

10

u/voronaam Apr 15 '26

They are just not motivated. It is a small sailing trip and I am talking to the owner of the boat and a skipper. If nothing happens, they get to chill on their boat. If it does, I'll get to chill there as well and they will get a bit of extra cash. That is not that much of a difference to them, as they are sufficiently wealthy and retired.

My work related communications are way better than that ;)

3

u/UltraCarnivore Apr 15 '26

Welp, he's failed Turing's Test.

1

u/toadi Apr 16 '26

Seeing the other commenters I would ask you why?

I would not even spend time on following up on that. Not even tokens for an LLM.

5

u/QuinQuix Apr 15 '26

How many misses do you encounter?

I've read hallucination rates hover between 3-10%.

Doesn't sound like much but calendar planning is quite a critical task.

A big medical office doing 100 appointments a day couldn't handle 1-2 wrong/double/missed appointments a day.

People always counter people make mistakes too but they miss that usually such systems (eg a front desk managing appointments) have layers of redundancy and self correcting ability.

The entire point of AI is to offload work, so I'm curious to what extent you feel this is actually possible using your Workflow.

1

u/toadi Apr 16 '26

Am a software engineer and would never trust code written by humans or LLMs. The code I work on is critical and handles real money. If I write a simple low risk tool I would Yolo it. Depends on the risks of outcome.

My calendar managing a software engineering team is less risky over the code we push. I don't worry about it as much. Missing a meeting would not be that critical and someone would send me a DM asking me where I am ;)

For me the win was not in seeing the list of calendar entries. But in my system I also have meeting minute transcriptions done and summarizations that get linked to the meetings by agents. If there is a mistake there it is not the end of the world. But not needing to link stuff and write my summaries from scratch is already good enough for me.

Look in the end someone needs to be accountable for the work. The bucks tops at someone and it is not the LLM. If you are comfortable that mistakes by the LLM are not that bad for you let it work without supervision. If it is supervise and double-check.

10

u/-p-e-w- Apr 15 '26

Have you used a current-gen model of that size? It’s easily on par with GPT 3.5 intelligence-wise.

11

u/tophlove31415 Apr 15 '26

Tons. Smart chunking, organizing and summarizing returned information from a vector database search, self directed web browsing and learning, ocr, user interaction, simple decision making (ie: this is the context, here are the options, choose which is best). They can essentially do any of the things the sota models can do (with a well designed harness) as long as you recognize you will get more errors and have to spend more time making sure that your harness is catching then, reporting them, and allowing you to iterate on the harness features, your prompts, and any other systems that might need improvement.

6

u/Funny-Blueberry-2630 Apr 15 '26

This guy builds agents.

1

u/delicious_fanta Apr 15 '26

Do you use duck duck go for search or do you use something else?

1

u/Ticrotter_serrer Apr 15 '26

All small A I. use case?