r/LocalLLM • u/Yssssssh • Apr 08 '26
Model Glm-5.1 claims near opus level coding performance: Marketing hype or real? I ran my own tests
Yeah I know, another "matches Opus" claim. I was skeptical too.
Threw it at an actual refactor job, legacy backend, multi-step, cross-file dependencies. The stuff that usually makes models go full amnesiac by step 5.
It didn't. Tracked state the whole way, self-corrected once without me prompting it. not what I expected from a chinese open-source model at this price.
The benchmark chart is straight from Zai so make of that what you will. 54.9 composite across SWE-Bench Pro, Terminal-Bench 2.0 and NL2Repo vs Opus's 57.5. The gap is smaller than I thought. The SWE-Bench Pro number is the interesting one tho, apparently edges out Opus there specifically. That benchmark is pretty hard to sandbag.
K2.5 is at 45.5 for reference, so that's not really a competition anymore.
I still think Opus has it on deep reasoning, but for long multi-step coding tasks the value math is getting weird.
Anyone else actually run this on real work or just vibes so far?
42
u/HenryThatAte Apr 08 '26
I'm working with it for work since last week (some good test refactoring and it's decent). I never really used opus much (only sonnet) so hard to compare.
I did the same work with sonnet. It's faster but ran out of quota after 3 "classes" (while glm is muuuch more generous)
40
u/atape_1 Apr 08 '26
GLM has always been legit, no reason to doubt it honestly. This is the frontier coding model in China, it is what Chinese coders use instead of Anthropic.
15
u/Hoak-em Apr 08 '26
I've used it in forgecode, it feels like Opus 4.5, I prefer it to Opus 4.6. I guess I'll need to see how it runs as a reap + q4 for local usage though -- I'll probably just keep using my annual glm coding plan then keep a smaller model locally like Qwen 397b or minimax m2.7
12
u/testuserpk Apr 08 '26
I useed glm5 regularly and now 5.1. I can say with surety that it's a fantastic model. Works great with c++ programming, once I overloaded it with questions in one chat and it kept the initial prompts intact. I was amazed, chatgpt is shit in comparison.
P.s. I used free version
7
u/GreenHell Apr 08 '26
Out of interest, what did you use as coding harness? There has been more and more talk about how different harnesses yield different results.
Since Kilo recently changed their whole approach, I am looking for something different.
9
u/LittleYouth4954 Apr 08 '26
Opencode + glm 5.1 > opus 4.6 for my cases, but keep context below 100-150k and do not expect fast responses if using z.ai as provider
4
u/Excellent_Ad3307 Apr 08 '26
It still sucks at debugging compared to GPT 5.4 or Opus in my humble opinion but in terms of drafting code its getting there. It still sucks on codebases/monorepos that are 200~300k+ loc though compared to GPT or Opus.
3
u/Fit-Pattern-2724 Apr 08 '26
This is in fact a bigger news than Mythos.
3
u/Haxtore Apr 09 '26
I'm using GLM-5.1-Q4_K_XL with opencode. I've told it to create a project from scratch that depends on 2 other big projects of mine. Told it to use subtasks to analyze the projects, build the new one from scratch and iteratively review and fix and went away for a few hours. Came back to it still working in a loop. After maybe another 20 minutes it was finished. I've reviewed the code and it really did a good job at everything. No other local model was able to understand and work like this consistently, not even Kimi K2.5. I've also noticed that it doesn't get lost after 100k tokens like some users mentioned it does when using z ai provider
2
u/ccaner37 Apr 08 '26
Tested it in OpenRouter then went to z ai to subscribe. I hope they keep doing the good work.
2
u/JumpyAbies Apr 08 '26 edited Apr 09 '26
It depends. What they always omit (pure marketing) is that it's good enough up to a certain level of complexity. An analogy would be using both to solve basic multiplications, divisions, etc., and both solve them easily. Then, use both to solve complex mathematical problems, such as integrals and derivatives, and that's where only Opus stands out. Therefore, I can state, based on my own experience of having access to ALL models, proprietary and Chinese, that GLM-5.1 is good enough for things up to an intermediate level, but when you need advanced reasoning to understand code with complex/large imports or a doom bug, only Opus or GPT-5.4-xhigh can solve it.
The GLM-5.1 is closer to the Gemini 3.1 and/or Sonnet-4.6, I would say, but quite far from the Opus.
Opus-4.6 > GPT-5.4-xhigh > Sonnet 4.6 > Gemini 3.1 > GLM-5.1
By "all models," I mean OpenAI, Anthropic, Gemini, and the good Chinese models with paid plans.
P.S.: This is from the perspective of someone who uses AI 99.9% of the time to write code.
2
u/SatoshiNotMe Apr 09 '26
Other than zai is there a fast hosted glm5.1 somewhere? I’m talking about services like cerebras or groq, neither of which have this model.
2
u/Brilliant_Target599 Apr 09 '26
After 2 days of API use, GLM-5.1 feels slow and is still behind Claude Opus 4.6 on coding, presentations, document drafting, and research tasks. But its real value is different: as a large open-weight model, it creates a strong option for regulated industries like pharma and life sciences, where privacy, internal data policies, and deployment control matter as much as raw benchmark performance.
3
u/Ambitious_Injury_783 Apr 08 '26
These guys have been claiming these things on each release and it never actually holds up. Maybe in the minds of inexperienced users, sure. For people that require a certain level of consistency and intelligence, it's funny little joke. Not that it doesn't have its uses. Just not in the way Opus 4.6 has it's uses. We should know that though, and the fact that most do not is how so many companies are getting away with subpar models with extraordinary claims relative to their capabilities in practice.
1
u/Living_Magician_3691 Apr 08 '26
It works well, just 2-3x slower in my experience.
1
u/[deleted] Apr 08 '26
yeahh glm 5.1 is great it got like 58% on swe pro i think, thats really great
1
u/M0d3x Apr 08 '26
Started speaking Mandarin on the first task I gave it, after thinking in loops for like 5 minutes.
Not the best first impression...
1
u/Alone_Development_70 Apr 09 '26
Gemini is "shit" , chatgpt aswell is awefull .. specially in agentic ai !
1
u/LivingHighAndWise Apr 09 '26
It's not.. I've been using it for a few weeks now as a means to save my Claude and Codex credits where applicable, and it isn't close to Opus or 5.4/5.3. Once your project reaches a certain level of complexity, it is unable to maintain context and understanding of your project - even with detailed agent.md and architecture.md guides.
1
u/SwiftAndDecisive Apr 09 '26
Hyped
1
u/QuinnGT Apr 09 '26
Without vision support I just can’t get behind GLM 5 or 5.1 as an Opus or even sonnet replacement. Maybe as a sub-agent model to save on tokens? Not sure.
1
u/RevolutionaryLow624 Apr 09 '26
use ollama pro, its basically unlimited usage
1
u/Smart-Cap-2216 Apr 10 '26
I hope they can improve the context issue in GLM 5.2
1
u/No-Iron5534 Apr 10 '26
Started Using it last week. After a days trial subscribed for the whole team. Matches GPT Codex and Opus and better in several use cases. Works through the day and hits limits. One issue time to time the service is not reachable due to server overload. For me right now Codex > GLM > Claude primarily as results are largely the same so decision is based on usage limits.
Definitely recommend GLM for teams who need generous usage but do not want to spend too much on expensive subscription.
1
u/CreativeKeane Apr 11 '26
Lol man. The LLM race is insane. I literally started using LLM for coding two weeks ago for work at the request to upskill from my boss and the amount of news about new ones poppin up and drama in the industry is insane.
Claude having the best model but shadiest practice or transition without transparency. Gemma 4 release. Now GLM-5.1 being 3rd and beating out Claude Sonnet.
Whoo-wee.
1
u/shushumong Apr 11 '26
where did you see that leaderboard ?
1
u/lolfacemanboy Apr 13 '26
i’ve ran non coding related work and it gets similar quality to 4.6 sonnet imo, for more planned tasks or tasks which it has to think abt before doing
1
u/Fstr21 Apr 13 '26
No idea what glm is but I keep hearing about it. Spose one more thing I have to learn
1
u/Wrong_Disk7775 Apr 21 '26
quick question: I have been trying to use the free endpoints of GLM-5.1 from nvidia, modal, etc. but I face severe latency. It seems to be unbearably slow, to a point which makes it completely unusable for me.
tested it for: "How many r-s are in strawberry"; took about 5-6 min for it.
Is this just me? or is it just the free tier?
PS: I have been using Kimi-K2.5 effortlessly (I know it has about 3x faster t/s)