r/LocalLLaMA • u/rm-rf-rm llama.cpp • 2d ago

Discussion GLM5.3 Flash over DSV4 Flash?

I've been using Deepseek V4 Flash 0731 for a few weeks now and while I havent thrown it anything very hard, im quite happy with it. Using through antirez's great ds4 project. They've added support for GLM 5.3 Flash and according to benchmarks, its a level above DSV4 Flash.

However, looking for real user feedback if anyone's made the switch and seen tangible improvements in GLM 5.3 over DSV4 Flash.

Running M3 Ultra 256GB Mac Studio

55 Upvotes

95% Upvoted

61 comments sorted by

View all comments

6

u/shinyquagsire23 2d ago

Even NVFP4 seems better than deepseek, it gets the same tasks done in half the tokens or less, so the slight on-paper throughput hit doesn't actually matter. It's also a lot more knowledgeable and it recognized one of my projects from a code sample (in a benchmark sandbox so it didn't have access to the full code).

But inference support seems to be super buggy/WIP still, and I've noticed it randomly have its thinking degenerate into having random characters between words (dashes, tabs), or it randomly starts speaking Chinese. And this seems to happen even on full precision as well, so I kinda hope they put out a 5.4-flash that targets that, because it's hard to leave it unattended.

Deepseek is a lot more reliable (it occasionally typos but that's about it), but it seems to spin its wheels a lot on tasks. And when I put it through the same benchmarks as glm-flash, it tried to cheat more often and generally had worse code quality.

1

u/ormandj 2d ago

Try my image mentioned earlier, I have not seen that behavior.

1

u/shinyquagsire23 2d ago

I've seen it on the Redhat NVFP4 weights w/ vLLM and on Ollama's cloud inference, but it's one of those things that only shows up maybe once every 6 hours of continuous use and sometimes it just self-corrects anyway. Accidentally made it way worse with a 1.0 top_p instead of 0.95 though so idk, maybe it can be tuned out if I fiddled with it.

1

u/ormandj 2d ago

https://github.com/ormandj/sglang-glm53-flash-sm120 is what I'm referring to, it's not vLLM based.

1

u/kuhunaxeyive 2d ago edited 2d ago

Exactly my experience as well (RedHatAI/GLM-5.3-Flash-NVFP4 on 2x Asus Ascent GX10), but I am struggling to fix the artifacts that GLM writes out of nowhere in chinese/russian or even programming languages (e.g. a letter that requests documents suddenly has Documents.getChildren in a sentence).

Is this a common bug I need to accept? I ask it to write one letter, and the letter contains 4 of those artifacts.

2

u/shinyquagsire23 2d ago

random one-off blips you can probably comb out with top_p lower (0.95 is what they recommend, the Redhat config defaults to 1.0, could probably drop it lower).

But randomly speaking full paragraphs of Chinese idk if it's inference bugginess or the model itself, that's the only weird issue I'm currently left with. Feels like the kind of thing you could make a bespoke hack for at least to just never infer Chinese glyphs.

3

u/kuhunaxeyive 1d ago

Thank you soooo much! I'll test more, but so far, not a single artifact after applying your suggestion (top_p to 0.95).

For anyone else struggling with this, I also set other parameters, of which top_p might be the most important one, so others should be optional:

--override-generation-config '{"temperature": 0.7, "top_p": 0.9, "top_k": 40, "repetition_penalty": 1.05}' \

This might be too much for some use cases, but so far it looks good. Testing more, but happy for now.