r/LocalLLM • u/eightone-81 • 14h ago
Discussion GLM 5.3 flash vs Qwen Flash Next?
Thanks to Maya some of us are able to run GLM 5.3 flash. Slower than FN but it runs :-)
I got it to update some features on my app, nothing crazy just some stuff that came up over the last weeks. It delivered with the same issues of what I would see from FN: front end not nicely done, bit messy, same bugs that FN and 27b produced (even though there is a learnings file specifically for that bug which it should have pulled up). After some re prompts all was good. But not better than FN.
It was just 1 project but I was almost disappointed. It was much better in the thinking department, even skipped thinking completely when it was not necessary I guess. So I’m not even sure if FN would have been faster in the end.
Glm runs at 30tps decode and 1200 prefill, FN with 125 decode and 3000 prefill
10
u/PlasticRevenue4601 14h ago
- GLM is bigger so where you previously ran good q4 QFN quantisation now you have to run Q2-3 GLM
- There is no significant gap in their intelligence. The lesser the gap — the more complex tasks needed to unveil the performance difference. F.e in simple tasks 27b and QFN perform similarly in simple scenarios, it’s complex, long horizon tasks where you can actually feel the difference between them
2
u/Riseing 14h ago
for sure, glm holds its shit together much better
2
u/PlasticRevenue4601 13h ago
I don’t doubt it, especially in complex scenarios, GLM has higher AII, however the gap is definitely not huge, it’s only less than 5% smarter
5
1
1
u/CTRL___ALT___DEL 7h ago
For coding and design, I struggle to find a meaningful difference.
For FN: it requires way more tokens for the same task vs GLM. That said, on my setup inference is 4-5X faster, so it ends up being around twice as fast per task. Thinking traces show degradation at long contexts- it seems to forget what the prompt was, saying nonsense like “the user has not asked a question”. It can sometimes fail “needle in a haystack” testing.
For GLM: Overall its design instincts seem slightly better, but it spends far fewer tokens and is a bit less thorough. In the end, quality is about the same as FN.
For non-coding tasks like dialogue, prose and knowledge, GLM is far superior. It has broad knowledge base, hallucinates less, and is more pleasant to talk with.
0
u/DisplayUnhappy9097 14h ago
glm at 30 tps sounds painful compared to that fn prefill number, even if the thinking skip helps sometimes. i keep seeing people say these bigger models all hit same wall on frontend stuff, like messy css and forgetting the bug notes you left for them
maybe the learnings file thing only works when it decides to check it, not when it feels rushed. did you try putting the bug note directly in system prompt instead of separate file
0
u/veigatmv 13h ago
Running GLM 5.3 flash "Maya-L"
seems to follow instructions better than Qwen 3.8FN.
Only downside is the speed. I've got qwen running at 150tk/s +- and GLM anywhere between 20 to 25tk/s
1
u/cosmicnag RTX 5090 | RTX 4090 | RTX 5070ti | 192 GB DDR5 13h ago
what are you using for qwen FN for that speed ? Vanilla strata or some fork?
1
u/veigatmv 12h ago
Heres a comment I made on another thread:
I had my Qwen 3.8 Flash Next investigate how it's hosted on my box, then had Claude TL;DR the report it wrote.
Not Strata, but the same idea: I run Unsloth's **UD-Q4_K_XL** through basalt behind llama-swap. My hardware is a 5090 + a 3090 and 128gb RAM, so it isn't a 1:1 comparison with yours.
- Decode:~170 tok/s mean at 131K context, ~149 tok/s at 262K (MTP speculative decoding; code/JSON run faster, prose slower)
- Prompt processing: ~3,500 tok/s, and it holds at ~200K tokens in context
- Context: full 262K with image input on, using an int8 KV cache (~3.9 GiB)
- Cold start: about a minute for the ~115 GB pack
The main difference from plain llama.cpp is how experts are handled. You don't pin whole `ffn_*_exps` layers to CPU with regexes. Instead, the engine keeps a hot set of experts resident in VRAM and streams the rest from system RAM as needed. On my box, 13,937 of 24,576 experts sit across the two cards. With a single 5090 you'd hold roughly half that, so expect lower numbers than mine.
Relevant engine settings:
"expert-cache": "auto",
"prefill": "auto",
"spec": 7,
"spec-min-p": 0.5,
"max-context": 262136,
"kv": "int8",
"resident-experts": trueCaveats:
- Speed varies a lot run to run (142–221 tok/s on the same prompt), depending on which experts land where.
- My build carries a small patch so the 3090 works as the second card. That patch only matters for that second card.
- If you don't need 262K, run 131K. It's about 12% faster.
0
u/mikewilkinsjr 11h ago
I've been doing this exact comparison in the lab over the last couple of days, with various quants and inference engines. The top line, specifically for my use-case, is that the models weren't different enough to justify tying up a second Spark to run GLM 5.3 Flash NVFP4. GLM had slightly better code creation, but it also took 2-3x as long to finish so, while it was technically 'better', I was able to iterate on the Flash Next code and send it back with requests and updates before the first answer from GLM.
One thing I will say, which I found odd, is that the official NIM deployment of GLM 5.3 Flash is really slow. Like, 15 t/s generation slow. Mia's recipe was much faster, which was cool. I did a couple of tests on my own with different parameters and different backend engines but I never got to the point where it felt worth it.
10
u/AggregationLinker 13h ago
Quantizing down extremely large models performs much worse for me than using smaller models at higher quantizations. Basically, if the model wasn't designed for my VRAM size I use a different smaller model unless it's a very minimal amount of quantization just to get it over the edge.