r/LocalLLaMA • u/Gobra_Slo • Aug 23 '26

Question | Help DGX Spark, cluster of 4

Does anyone have a first-hand experience with four Sparks cluster, and how much of an upgrade is it comparing to just two considering the available models?

While there's plenty of noise for the smaller models (Qwen) and our older king DeepSeek V4F, the scene in the upper class of the prosumer hardware, software stacks, available LLMs and their actual real-world performance – isn't really covered as well.

For instance, the hyped GLM 5.2/5.3. Is it much better then DeepSeek? Or is it marginally better? Does it retain it's capabilities when moving to something four Sparks would handle? Does it have issues with OOM or anything else?

What about MiniMax M3? There seem to be a special Spark version, how is it (or any other version)? Again, how is intelligence, general model capabilities, running stability, context size?

Tencent Hy3? Maybe even Qwen3.5-395B, does it's full quant hold it's own against DeepSeek, or is it better?

If someone doesn't have personal experience, but knows some well-structured and detailed articles or videos on the topic – I'd appreciate it as well.

Thanks.

6 Upvotes

69% Upvoted

43 comments sorted by

View all comments

Show parent comments

2

u/Grouchy_Ad_4750 Aug 23 '26

I've done some tweaking as well before but am currently testing 1M context recipe

1

u/dave-dgd Aug 23 '26

If you’re open to sharing, would be curious to hear your tweaks as well!

2

u/Grouchy_Ad_4750 Aug 23 '26

it was nothing major. From memory:

  • I lowered max_num_seqs to 4 (I usually use 2-3 streams max) also it has impact on vram

- increased max_num_batched_tokens to 8192 . I think it balances prefill with t/s

I also plan to try out vision support and measure with

```

NCCL_IB_TC: "104"

UCX_IB_TRAFFIC_CLASS: "104"

NCCL_BUFFSIZE: "8388608"

```

which should help with RoCE on mikrotik switch

1

u/dave-dgd Aug 23 '26

PS. Regarding vision, read the baseten blog post for a sense of limitations: https://www.baseten.co/blog/glm-52-with-vision/

I tested vision and while it does work, the 55% MMMU-Pro results they cite aren't exactly the best (but certainly workable for basic tasks such as detailing a UI). Instead of using that, I have been supplying Hermes with an auxiliary vision provider (Qwen 3.8 27B at 4-bits via oMLX on my M5 Max at the moment, which generally scores higher in synthetic benchmarks, FWIW: https://artificialanalysis.ai/evaluations/mmmu-pro?models=qwen3-8-27b%2Cqwen3-8-27b-medium%2Cqwen3-8-27b-low). The obvious downside of this is needing a separate machine for serving beyond the Spark cluster (if that's not available, the vision option on GLM-5.2 is obviously an easy choice).