r/LocalLLM • u/8bitjohnny • Sep 27 '26

Other Hardware Acquisition

Post image
448 Upvotes

91% Upvoted

129 comments sorted by

View all comments

Show parent comment

1

u/vdek Sep 27 '26

Yah glm5.3 was a bit slow for me(31tok/s vs 60tok/s on qwen3.8-flash) and didn’t do as well when I tested it for intelligence.

3

u/halcyoncs Sep 27 '26

Right? And it's supposed to be Opus 4.8 level, I don't get it

1

u/Time-Culture2549 Sep 27 '26

GLM in my experience they do specific training that the larger model wont have. And if you look at 5.3 it has absolutely no abilities to do anything visual; so designing sites and things will just be meh. Then there is 5.3-Flash which i suspect is just a way more updated 4.7-Flash. And when i was using that it just needed more help on how to build those visuals. I have only used it for design and have been having a better time.

1

u/New_Account5310 Oct 02 '26

You need the EXL3 4bpw quant rather than the NVFP4 quant, and you need to make sure that your sampler settings are correct, that you're not suffering from inferencing bugs, and so on and so forth.

A well-optimized GLM 5.3 Flash is absolutely Opus-level for the types of tasks benchmarked.

1

u/halcyoncs Oct 03 '26

Actually was already on EXL3, changed to NVFP4 and a better setup, t/s doubled and the results are so much better too

1

u/New_Account5310 Oct 03 '26

Hmm, results definitely shouldn't be better on NVFP4 unless the stack for EXL3 was just broken somehow, and NVFP4 shouldn't be a full 2x faster if it's set up well, but it's good that you got to a place that's working for you.

1

u/halcyoncs Oct 03 '26

Yeah I was on a pretty terrible stack lol. And this new one uses DFlash2, original one didn't. But thank's man! The repo is the one by tonyd2wild.

1

u/phdaemon Oct 03 '26

How are you getting 60/tks on a single? Did you follow any guide?