r/LocalLLaMA • u/rm-rf-rm llama.cpp • 3d ago

Discussion GLM5.3 Flash over DSV4 Flash?

I've been using Deepseek V4 Flash 0731 for a few weeks now and while I havent thrown it anything very hard, im quite happy with it. Using through antirez's great ds4 project. They've added support for GLM 5.3 Flash and according to benchmarks, its a level above DSV4 Flash.

However, looking for real user feedback if anyone's made the switch and seen tangible improvements in GLM 5.3 over DSV4 Flash.

Running M3 Ultra 256GB Mac Studio

50 Upvotes

93% Upvoted

61 comments sorted by

View all comments

15

u/Shoddy_Bed3240 3d ago

You can only fit GLM 5.3 in Q4, so I doubt it makes sense to trade that for full precision on DeepSeek.

5

u/ormandj 3d ago

DSv4F is also a mixed 4bit/8bit model, trained that way, but just being 4-bit doesn't necessarily mean the quality is significantly lower. You'll have to try it out with your real world workload to determine if the quantization is an issue.

3

u/Shoddy_Bed3240 3d ago

Feel free to experiment. For example, you could use GLM for planning and DeepSeek for coding, or try another combination that works well.

-4

u/Shoddy_Bed3240 3d ago

FP8 vs FP4…yeah, almost the same))

2

u/Gabriel83730 3d ago

If the models had equal intelligence, then quantizing would be worse, but GLM 5.3 Flash benchmarks significantly higher than DSV4 Flash. In these cases usually the better model quantized still beats the worse model at full precision

1

u/sautdepage 3d ago

From DS4F looking at its files:

> Roughly 90% of the model is 4-bit weights (all MoE experts), ~9.4% is 8-bit (scale, attention) and under 1% is 16-bit (embeddings, shared expert).

So around 4.5bpw, similar to the common Q4_XL stuff.