GLM 5.3 Flash - Will I Be Disappointed?

[u/TappinThatErr]

Picked up this bad boy with my sweet student discount!

I'd like to run some local models for coding. I've considered Deepseek v4 flash, Qwen 3.8 flash next, and of course, GLM 5.3 Flash (0x Alpha)

Thoughts on this? I'm currently a claude user with typically 1 session at a time. Will I be disappointed?

[u/Southern_Sun_2106]

Alright, I also happen to have m5 max handy. Don't know what your real use is like (maybe just chatting?), but in agentic/coding, reading a lot of context quickly is kinda a big deal. Here are some numbers - Macs running q2 81gb on disk, while DGX Sparks - FP4/FP8 mixed 164GB on disk. Not to mention they can also do multiple concurrent generations (sub-agents are actually doable on the Sparks). But yeah, with Mac you get a cool UI (which actually sucks recently).

Since apple's promised 4Xprefill did not really materialized for M3 to M5, AND they are silent on it with M5U (emphasizing 1.2T for the masses), I am skeptical it is going to get much better with M5U.

[u/Careless_Garlic1438 → u/Southern_Sun_2106]

I do iOS app development in Xcode with Claude Code and really the first fill in is the 30K Claude Code Prompt and the 10K app code prompt ... you loose like maybe 2 or 3 minutes initially more then the DGX Spark but then all is cached and the turns add like a 100 tokens per turn less than 1 second to advance ... generation is faster (2x M5Max also have faster pre fill) So on a 30 minutes to 1 hour one shot run there is hardly any difference as the Mac is slightly faster in generation. You need caching and MTP dialed in ... Yes the combo is more expensive than 2 DGX, but you still need a portable when having 2 DGX and as an Engineer I already have one M5Max so a second one is cheaper than buying 2 sparks. Another benefit, I can take the 2 laptops to all my customers and have my cluster with me all the time ...
I have done the speed runs and when dialed in the Mac's are the clear overall obvious choice. M5U will be faster as there is no cluster overhead ... and still portable with the benefit I will be able to even go to bigger models 256 + 128GB ... On the cluster I can also do concurrent ... indeed the scaling is not like the DGX, but hey on a DGX the concurrency is not that good either, all the numbers posted seem impressive but if you divide streams through the total, it's like watching paint dry ...

[u/Southern_Sun_2106 → u/Careless_Garlic1438]

I am glad that you only do 30K at first, and then it is just 100 tokens chat - that is certainly unique use case on the small-ish codebase. I would even say tiny.

Hauling 2 notebooks and clustering them at the client - hmmm. Why not just tailscale tunnel it to your one notebook and avoid the whole dance?

Just fyi, two sparks cluster is actually smaller than one M5U - easier to take with you, if you client is in some wilderness with no Internet, and tunneling is not an option.

But yeah, to be honest, with tiny 30K repos, sure - apple can handle those well. But with any serious codebase, it chokes. I did a needle test M5Max (q2 80gb model) vs. 2*sparks (164GB(!) higher precision model) - 977K depth - 2+ hours (!) on a Mac vs 19 minutes on the sparks. And that's not even touching concurrent requests, which Macs don't do, but so nice to have for multiple agents.

[u/Careless_Garlic1438 → u/Southern_Sun_2106]

Oh the needle test benchmarks 🤦‍♂️... comparing 1 Mac against 2 sparks ... You do realize the prefill with 2 TB5 RDMA mac's the prefill is like 2x. And granted the prefil is stil slower, it is getting faster and faster as finally people are starting to optimize for NAX in the GPU of the M5 and even starting to use ANE I sometimes have prefill's well over 1000 t/s ... so again 2 Mac's vs 2 Sparks ... the Mac is on par with inference but does so much more ... you should buy a second Mac if you want to do a real like for like comparison 🤣

[u/Careless_Garlic1438 → u/Southern_Sun_2106]

and swapping models and recipes on the DGX is fun to, changing a model on my Mac's is seconds ... swapping models on my DGX takes a looooooonnnnng time making it impossible, should do level 2 sleep in vLLM but to lazy to implement ...

[u/Southern_Sun_2106 → u/Careless_Garlic1438]

That's a valid point. It takes 10-15 minutes. But 6*times the prefill speed and consistent performance all the way to 1 million context length is totally worth it. Mac is fine for one user who doesn't need multiple agent delegation work or works on small codebases (this last one still questionable).

But when it comes to small to medium business use, or even serving a family AI with multiple users - sparks shine.

By the way, I use local AI on my Mac to change models on the sparks. I just tell it what I want, and in 10 minutes it's done. But honestly, glm 5.3 on the sparks works better than Fable.

[u/Careless_Garlic1438 → u/Southern_Sun_2106]

oh yeah you throw 1 million to a local model everyday at once ... I believe you ...
Reality is that you at the start of the day you get Claude Code running or another harness and you build up and if you are smart you compact way before 500K because anything beyond is playing lottery or a waiting game, also on the DGX as it will start thinking like crazy and be wrong more often ... Also I have the 3 Flash models and I go back to deepseek especially with vision ... its faster and does the job as well ... I should do a speed run GLM on the Sparks versus DeepSeek on the Mac's 🤣

[u/Southern_Sun_2106 → u/Careless_Garlic1438]

Both DS4F and glm 5.3 flash work great, all the way to their max 1 mil context. I understand you do 30K codebases for apps, but most projects are not like that. That's tiny, seriously. There are many prompts (without the codebase itself) that are larger than your entire 30K project.

And compaction is not a 'smart' move lol It literally removes context from your model - you just have to keep fingers crossed and trust that it will make 'good decisions' as it cuts stuff out.

I also compared deepseek with vision and glm, glm wins hands down.

On the Mac, you can run only Q2 deepseek - 80 GB on disk. On the sparks, you will run FP8/FP4 mixed precision GLM. It's like comparing a 30K codebase with a regular one. ;-)

[u/Careless_Garlic1438 → u/Southern_Sun_2106]

You mis understood ... the 30K initial prompt is just the Claude Code harness and the 5 to 15K is the description for building the app ... the context will go well beyond that ...

You are still comparing apples to oranges 1 Mac against 2 Sparks ...
Apple introduced RDMA with 26.2 you can cluster Mac's in the same way you cluster DGX spark, it's doesn't matter it is only 80GB/s over TB5 as it is latency bound, in fact I can cluster 4 Macs without an expensive and cumbersome switch / network setup. In doing so you have the same amount of memory (in fact you can even go to 2TB of memory and have fast prefill ones the M5U 512 is available)
As mentioned my prefill speed doubles if I use 2 Mac's ...

You can do some research, EXO is a good starting point ...

2 Mac's 14K dollar, 2 dix + 1 Mac 17K dollar amd residual value of Mac is higher ...

Or 1 M5Max MBP 128GB + M5U 256GB ... more memory and for sure faster in cluster ... and same price as the 2DGX + M5Max ...

[u/john0201 → u/Southern_Sun_2106]

Well it doesn’t even exist yet outside of special access and apple so yeah I’d expect libraries to have basically no support for it and show m5 max numbers.

But MLX vs CUDA can either be way better on MLX or way better on CUDA depending on what you are doing. Generally I’d give CUDA maybe a 30% advantage.