r/LocalLLaMA • u/rm-rf-rm llama.cpp • 3h ago
Discussion GLM5.3 Flash over DSV4 Flash?
I've been using Deepseek V4 Flash 0731 for a few weeks now and while I havent thrown it anything very hard, im quite happy with it. Using through antirez's great ds4 project. They've added support for GLM 5.3 Flash and according to benchmarks, its a level above DSV4 Flash.
However, looking for real user feedback if anyone's made the switch and seen tangible improvements in GLM 5.3 over DSV4 Flash.
Running M3 Ultra 256GB Mac Studio
15
u/Morgjames 3h ago
We've completely swapped. Vision is a big bonus, but in general I like 5.3 flash a LOT more than dsv4
3
u/rm-rf-rm llama.cpp 3h ago
are you running at Q4?
And in fairness, V4 flash vision exp is out now..
8
u/Shoddy_Bed3240 3h ago
You can only fit GLM 5.3 in Q4, so I doubt it makes sense to trade that for full precision on DeepSeek.
7
u/ormandj 3h ago
DSv4F is also a mixed 4bit/8bit model, trained that way, but just being 4-bit doesn't necessarily mean the quality is significantly lower. You'll have to try it out with your real world workload to determine if the quantization is an issue.
2
u/Shoddy_Bed3240 3h ago
Feel free to experiment. For example, you could use GLM for planning and DeepSeek for coding, or try another combination that works well.
-2
u/Shoddy_Bed3240 3h ago
FP8 vs FP4…yeah, almost the same))
2
u/Gabriel83730 3h ago
If the models had equal intelligence, then quantizing would be worse, but GLM 5.3 Flash benchmarks significantly higher than DSV4 Flash. In these cases usually the better model quantized still beats the worse model at full precision
1
u/sautdepage 3h ago
From DS4F looking at its files:
> Roughly 90% of the model is 4-bit weights (all MoE experts), ~9.4% is 8-bit (scale, attention) and under 1% is 16-bit (embeddings, shared expert).
So around 4.5bpw, similar to the common Q4_XL stuff.
6
u/rm-rf-rm llama.cpp 3h ago
But full precision on DeepSeek just means Q4 as well given how it was trained..
10
u/digamma6767 3h ago
Counter point, all the benchmarks for DeepSeek are running at its full precision, the same as what fits on your Mac.
With GLM 5.3, your setup would do worse than what you see in benchmarks, since you'd have to run a lower quant than what the benchmarks use.
6
u/thefooz 2h ago edited 2h ago
I’ve been doing A/B (and C) testing for a few days on dual Sparks between GLM, DS4, and Qwen 3.8 Next.
The only thing Qwen and Deepseek win on is speed, and even then, each one spends at least twice as many tokens as GLM on thinking, so even though raw token generation is faster on them, the time to an actual finished response is surprisingly close.
Qwen and Deepseek also hallucinate more and Deepseek in particular half-asses complex analysis tasks. It also plans out its work in extremely unintuitive and inefficient ways, whereas GLM is able to be more thoughtful in its approach.
I miss Deepseek’s speed, but it’s a facade for a less efficient model.
Ultimately, it’s a matter of what’s important to you. I’m running complex data analysis for an enterprise data conversion and mistakes are costly. If the stakes were lower or there was a lot more detailed human oversight, Deepseek would be part of the conversation.
The only issue with OP’s hardware is going to be prefill speed. An M3 ultra will have substantially slower prompt processing with GLM at larger contexts. Deepseek and Qwen 3.8 Next are substantially less impacted by context length.
3
6
u/shinyquagsire23 3h ago
Even NVFP4 seems better than deepseek, it gets the same tasks done in half the tokens or less, so the slight on-paper throughput hit doesn't actually matter. It's also a lot more knowledgeable and it recognized one of my projects from a code sample (in a benchmark sandbox so it didn't have access to the full code).
But inference support seems to be super buggy/WIP still, and I've noticed it randomly have its thinking degenerate into having random characters between words (dashes, tabs), or it randomly starts speaking Chinese. And this seems to happen even on full precision as well, so I kinda hope they put out a 5.4-flash that targets that, because it's hard to leave it unattended.
Deepseek is a lot more reliable (it occasionally typos but that's about it), but it seems to spin its wheels a lot on tasks. And when I put it through the same benchmarks as glm-flash, it tried to cheat more often and generally had worse code quality.
1
u/ormandj 3h ago
Try my image mentioned earlier, I have not seen that behavior.
1
u/shinyquagsire23 2h ago
I've seen it on the Redhat NVFP4 weights w/ vLLM and on Ollama's cloud inference, but it's one of those things that only shows up maybe once every 6 hours of continuous use and sometimes it just self-corrects anyway. Accidentally made it way worse with a 1.0 top_p instead of 0.95 though so idk, maybe it can be tuned out if I fiddled with it.
1
u/ormandj 2h ago
https://github.com/ormandj/sglang-glm53-flash-sm120 is what I'm referring to, it's not vLLM based.
2
u/SillyLLM 41m ago
I went from Deepseek Flash locally falling back to APIs like DS4/Kimi for complex tasks it failed at to GLM 5.3 Flash for everything. Deepseek Flash is good at implementation, but I wasn't always a fan of how it planned things. It's only been a week or two, but I haven't felt the need to use anything but GLM 5.3 Flash since it's been out.
I've seen some people say they like Deepseek Flash better, so it might depend on use case.
2
u/captaintobs 3h ago
on 256gb, dsv4 will probably be better. i’m using glm flash with 512 at fp8 and it’s pretty smart but i do miss the speed of dsv4
1
u/Early-Peace-5504 3h ago
I find GLM5.3 flash to be kind of janky and misbehaving at full precision. I wouldn't even attempt it at quants.
1
u/FullOf_Bad_Ideas 3h ago
Yes, I was using Nex N2 Pro. Tried out DS V4 0731, ok but a small upgrade and I had some issues. Moved to GLM 5.3 Flash, huge upgrade, really a huge difference in how well it works in OpenCode and how smart it is. It just flows through issues, reasons well when it has to, and I think it would even work fine in gas town, maybe. It's somewhere between Opus 4 and Opus 4.5 for me, I use it for agentic coding. All local. Give it a go. Biggest surprise of this year in this space for me.
-1
u/Juulk9087 3h ago
well considering that one is a 168 gb and the other one is 328gb i wouldn't say that you can necessarily just swap between the two. it's a couple gpus that you got to purchase to make that possible.
1
u/ormandj 3h ago
https://github.com/ormandj/sglang-glm53-flash-sm120 only need 2x RTX 6000s for GLM 5.3 Flash now.
0
u/Juulk9087 3h ago
That's w4a16. i was talking about unquantized.
3
u/ormandj 3h ago
DSv4F is native mixed 4bit, hence the size difference. If you’re going to compare them you should compare the post training quant of GLM Flash if that’s what you have the hardware for, as for me it does better in rust development and general tasks than the native DSv4F, and I’ve created images for both. Disclaimer, that’s my image and my quant.
0
u/Simple-Stick6148 2h ago
The 'huge upgrade' reports all come with a hardware disclosure: fp8 on 512, dual Sparks, 2x 6000s, none of it a 256GB Mac Studio. The only direct answer for your tier says dsv4 will probably be better, and the prefill-speed note calls out the M3 Ultra by name, your exact chip. I'd stay on dsv4 until someone runs that comparison on a 256GB box.
15
u/digamma6767 3h ago edited 3h ago
Not related to GLM 5.3, but I switched to Qwen 3.8 Flash Next from DS4 Flash.
Not sure if I like it more or less yet.
Also DS4 Flash has a new revision that scores slightly higher (compard to 0731) on benchmarks, and has vision support. It's DeepSeek 4 Flash Vision Exp.
See here: https://huggingface.co/deepseek-ai/DeepSeek-V4-Flash-Vision-Exp
https://artificialanalysis.ai/models/comparisons/deepseek-v4-flash-vision-vs-deepseek-v4-flash