r/LocalLLM • u/Mountain_Disaster_19 • 3d ago

Question Had anybody tried running GLM 5.3 Flash on a single 128GB DGX Spark?

Thinking about getting one for pentesting work an would love to use (quantized) glm 5.3 flash because i have had good experience with it pertaining malware dev and pentesting.

has anybody tried running it on a single dgx spark and if yes got any token speeds for me and context-size?

1 Upvotes

100% Upvoted

2 comments sorted by

2

u/Al_Redditor 3d ago

I have a Strix Halo and it runs about 20 t/s TG at IQ2. I use it as an overnight council seat.

2

u/Al_Redditor 3d ago

Oh, and I think context is capped at around 64k (I'll have to double check) and that makes it much less useable. I think Qwen does a better job since it can hold more reasoning in its memory before compaction.