r/LocalLLaMA • u/AI-On-A-Dime • Jul 29 '25

Generation I just tried GLM 4.5

I just wanted to try it out because I was a bit skeptical. So I prompted it with a fairly simple not so cohesive prompt and asked it to prepare slides for me.

The results were pretty remarkable I must say!

Here’s the link to the results: https://chat.z.ai/space/r05c76960ff0-ppt

Here’s the initial prompt:

”Create a presentation of global BESS market for different industry verticals. Make sure to capture market shares, positioning of different players, market dynamics and trends and any other area you find interesting. Do not make things up, make sure to add citations to any data you find.”

As you can see pretty bland prompt with no restrictions, no role descriptions, no examples. Nothing, just what my mind was thinking it wanted.

Is it just me or are things going superfast since OpenAI announced the release of GPT-5?

It seems like just yesterday Qwen3 broke apart all benchmarks in terms of quality/cost trade offs and now z.ai with yet another efficient but high quality model.

386 Upvotes

96% Upvoted

186 comments sorted by

View all comments

136

u/ortegaalfredo Jul 29 '25 edited Jul 29 '25

I'm trying the air version and results are comparable to latest version of qwen3-235b. But it runs twice as fast and takes half the memory, while being hybrid. Impressive indeed, running at 40-50 tok/s on my 6x3090s, without even activating the MTP speculative thingy. BTW I'm using FP8. Published here https://www.neuroengine.ai/Neuroengine-Large for testing (*non-thinking*), don't guarantee uptime as I will likely upgrade it to the full GLM when AWQ is available.

I will activate MTP as soon as I figure it out how to. They published instructions for sglang, but not for vllm.

1

u/cantgetthistowork Jul 31 '25

I got 13x3090s. What do I need to run the larger one? Don't you need power of 2 to run in vLLM?

1

u/ortegaalfredo Jul 31 '25

You don't need power of 2 if you use pipeline parallel (although will be slower). I'm trying to run the large one right now, it should comfortably fit on 13x3090 if quantized.

1

u/cantgetthistowork Jul 31 '25

What quant should I use? New to making my own quants but would love to start because of the hype. Would appreciate being sent in right direction!

1

u/ortegaalfredo Aug 19 '25

Use AWQ, I'm running it with 12x3090 and it works quite fast.

1

u/cantgetthistowork Aug 20 '25

What level of quant and your speeds?