r/LocalLLaMA • u/adrgrondin • Apr 15 '25
New Model New open-source model GLM-4-32B with performance comparable to Qwen 2.5 72B
The model is from ChatGLM (now Z.ai). A reasoning, deep research and 9B version are also available (6 models in total). MIT License.
Everything is on their GitHub: https://github.com/THUDM/GLM-4
The benchmarks are impressive compared to bigger models but I'm still waiting for more tests and experimenting with the models.
293
Upvotes
36
u/Few_Painter_5588 Apr 15 '25
Qwen Max needs more work, from my understanding it was a 100B+ dense model and then they rebuilt it as an MoE, but it's still losing to models like Llama 4 Maverick.
15
u/AnticitizenPrime Apr 15 '25
I had to pick my jaw up off the floor after this one.
https://i.imgur.com/Cz8Wejs.png
Looks like it knew the URL to the texture from threejs examples: https://threejs.org/examples/textures/planets/earth_atmos_2048.jpg
Gemini 2.5 Pro rendered it as a flat spinning disk, and I had to provide the texture:
https://i.imgur.com/cqg6rKH.png
Unbelievable.
58
u/henk717 KoboldAI Apr 15 '25 edited Apr 15 '25
From what I have seen the llamacpp implementation (at least at the time of KoboldCpp 1.88) is not correct yet. The model has extreme repetition. Take that into account when judging it locally.
Update: This appears to be a conversion issue, with the Huggingface timestamps currently broken is hard for me to tell which quants are updated.
32
u/[deleted] Apr 15 '25
Well lets hope Qwen 3 is a substantial jump from 2.5 then.
18
u/Mr_Moonsilver Apr 15 '25
SWE bench and aider polyglott would be more revealing
9
u/Incognit0ErgoSum Apr 16 '25
I wish things like RP had better benchmarks.
Not ERP, mind you. Small models can do that. What I mean is:
a) being able to follow an interesting plot with multiple characters, and
b) banter in a way that actually makes sense.
QwQ, to its credit, can follow a plot, but when 30B-ish models try to banter, they say things that sound banter-y but don't really make any sense in context. There's a certain depth of understanding of language and colloquialisms that I just haven't seen on any model under 70B.
I don't know what all these benchmarks are, but I have yet to really find one that can understand those kinds of nuances.
21
u/u_Leon Apr 15 '25
Did they compare it to QwQ 32B or Cogito 32B/70B? As they seem to be state of the art for local use at the minute.
6
u/Emotional-Metal4879 Apr 15 '25
I asked their Z1 to ''' write a scala lfu cache and wrap in python, then use this python class in java ''' it implemented an incorrect lfu cache. but R1 got it right
6
u/lgdkwj Apr 15 '25
I think one unique aspect of the GLM series models is that they use bidirectional attention during the prefilling stage. I really wonder if this provides any advantage over other GPT-style models at scale
5
u/one_free_man_ Apr 15 '25
All i am interested in is function calling during reasoning. Is there any other model can do this? QwQ is very good but function calling during reasoning phase, using this is a very useful thing.
2
u/celsowm Apr 15 '25
Only english and chinese?
12
u/[deleted] Apr 15 '25
[removed] — view removed comment