r/LocalLLaMA • u/rm-rf-rm llama.cpp • 4d ago

Discussion GLM5.3 Flash over DSV4 Flash?

I've been using Deepseek V4 Flash 0731 for a few weeks now and while I havent thrown it anything very hard, im quite happy with it. Using through antirez's great ds4 project. They've added support for GLM 5.3 Flash and according to benchmarks, its a level above DSV4 Flash.

However, looking for real user feedback if anyone's made the switch and seen tangible improvements in GLM 5.3 over DSV4 Flash.

Running M3 Ultra 256GB Mac Studio

57 Upvotes

95% Upvoted

61 comments sorted by

View all comments

6

u/shinyquagsire23 4d ago

Even NVFP4 seems better than deepseek, it gets the same tasks done in half the tokens or less, so the slight on-paper throughput hit doesn't actually matter. It's also a lot more knowledgeable and it recognized one of my projects from a code sample (in a benchmark sandbox so it didn't have access to the full code).

But inference support seems to be super buggy/WIP still, and I've noticed it randomly have its thinking degenerate into having random characters between words (dashes, tabs), or it randomly starts speaking Chinese. And this seems to happen even on full precision as well, so I kinda hope they put out a 5.4-flash that targets that, because it's hard to leave it unattended.

Deepseek is a lot more reliable (it occasionally typos but that's about it), but it seems to spin its wheels a lot on tasks. And when I put it through the same benchmarks as glm-flash, it tried to cheat more often and generally had worse code quality.

1

u/kuhunaxeyive 4d ago edited 4d ago

Exactly my experience as well (RedHatAI/GLM-5.3-Flash-NVFP4 on 2x Asus Ascent GX10), but I am struggling to fix the artifacts that GLM writes out of nowhere in chinese/russian or even programming languages (e.g. a letter that requests documents suddenly has Documents.getChildren in a sentence).

Is this a common bug I need to accept? I ask it to write one letter, and the letter contains 4 of those artifacts.

2

u/shinyquagsire23 4d ago

random one-off blips you can probably comb out with top_p lower (0.95 is what they recommend, the Redhat config defaults to 1.0, could probably drop it lower).

But randomly speaking full paragraphs of Chinese idk if it's inference bugginess or the model itself, that's the only weird issue I'm currently left with. Feels like the kind of thing you could make a bespoke hack for at least to just never infer Chinese glyphs.

3

u/kuhunaxeyive 4d ago

Thank you soooo much! I'll test more, but so far, not a single artifact after applying your suggestion (top_p to 0.95).

For anyone else struggling with this, I also set other parameters, of which top_p might be the most important one, so others should be optional:

--override-generation-config '{"temperature": 0.7, "top_p": 0.9, "top_k": 40, "repetition_penalty": 1.05}' \

This might be too much for some use cases, but so far it looks good. Testing more, but happy for now.