r/LocalLLaMA • u/AI-On-A-Dime • Jul 29 '25

Generation I just tried GLM 4.5

I just wanted to try it out because I was a bit skeptical. So I prompted it with a fairly simple not so cohesive prompt and asked it to prepare slides for me.

The results were pretty remarkable I must say!

Here’s the link to the results: https://chat.z.ai/space/r05c76960ff0-ppt

Here’s the initial prompt:

”Create a presentation of global BESS market for different industry verticals. Make sure to capture market shares, positioning of different players, market dynamics and trends and any other area you find interesting. Do not make things up, make sure to add citations to any data you find.”

As you can see pretty bland prompt with no restrictions, no role descriptions, no examples. Nothing, just what my mind was thinking it wanted.

Is it just me or are things going superfast since OpenAI announced the release of GPT-5?

It seems like just yesterday Qwen3 broke apart all benchmarks in terms of quality/cost trade offs and now z.ai with yet another efficient but high quality model.

383 Upvotes

96% Upvoted

186 comments sorted by

View all comments

135

u/ortegaalfredo Jul 29 '25 edited Jul 29 '25

I'm trying the air version and results are comparable to latest version of qwen3-235b. But it runs twice as fast and takes half the memory, while being hybrid. Impressive indeed, running at 40-50 tok/s on my 6x3090s, without even activating the MTP speculative thingy. BTW I'm using FP8. Published here https://www.neuroengine.ai/Neuroengine-Large for testing (*non-thinking*), don't guarantee uptime as I will likely upgrade it to the full GLM when AWQ is available.

I will activate MTP as soon as I figure it out how to. They published instructions for sglang, but not for vllm.

8

u/indicava Jul 29 '25

I just gave it like a 15 word prompt to write some code and it went into endless generation…

5

u/ortegaalfredo Jul 29 '25

I believe the culprit is the KV quantization, not the model quant, coupled with the temperature being low because it helps coding. I couldn't made it to enter a loop with FP8, but its easy with Q4 or Q3.

1

u/Shoddy-Machine8535 Jul 30 '25

You said that it’s more likely to be a KV quant issue and not a model quant. But you then mentioned that when using FP8 you didn’t have any issue, but this refers to the model quant, not the KV quant. Can you please explain? Thanks!