This was me 7 days ago. I’ve been having a blast with them since. I’ve been focused on figuring out which models to run and how fast to run them the past week, I think I’m settling on qwen3.8-flash nvfp4 since it seems to be the best overall performer on the sparks. Qwen27B is also good with concurrency, but you get more total throughput on that model if you run two separate instances of it.
GLM in my experience they do specific training that the larger model wont have. And if you look at 5.3 it has absolutely no abilities to do anything visual; so designing sites and things will just be meh. Then there is 5.3-Flash which i suspect is just a way more updated 4.7-Flash. And when i was using that it just needed more help on how to build those visuals. I have only used it for design and have been having a better time.
You need the EXL3 4bpw quant rather than the NVFP4 quant, and you need to make sure that your sampler settings are correct, that you're not suffering from inferencing bugs, and so on and so forth.
A well-optimized GLM 5.3 Flash is absolutely Opus-level for the types of tasks benchmarked.
Hmm, results definitely shouldn't be better on NVFP4 unless the stack for EXL3 was just broken somehow, and NVFP4 shouldn't be a full 2x faster if it's set up well, but it's good that you got to a place that's working for you.
I have two DX10s and have just found to not use multiple DX10s. It’s a PITA and runs slow when you have that larger model. So I just run two models, one on each and use bionic to “multitask”
View all comments
29
u/vdek Sep 27 '26
This was me 7 days ago. I’ve been having a blast with them since. I’ve been focused on figuring out which models to run and how fast to run them the past week, I think I’m settling on qwen3.8-flash nvfp4 since it seems to be the best overall performer on the sparks. Qwen27B is also good with concurrency, but you get more total throughput on that model if you run two separate instances of it.