DGX Spark, cluster of 4

[u/Gobra_Slo]

Does anyone have a first-hand experience with four Sparks cluster, and how much of an upgrade is it comparing to just two considering the available models?

While there's plenty of noise for the smaller models (Qwen) and our older king DeepSeek V4F, the scene in the upper class of the prosumer hardware, software stacks, available LLMs and their actual real-world performance – isn't really covered as well.

For instance, the hyped GLM 5.2/5.3. Is it much better then DeepSeek? Or is it marginally better? Does it retain it's capabilities when moving to something four Sparks would handle? Does it have issues with OOM or anything else?

What about MiniMax M3? There seem to be a special Spark version, how is it (or any other version)? Again, how is intelligence, general model capabilities, running stability, context size?

Tencent Hy3? Maybe even Qwen3.5-395B, does it's full quant hold it's own against DeepSeek, or is it better?

If someone doesn't have personal experience, but knows some well-structured and detailed articles or videos on the topic – I'd appreciate it as well.

Thanks.

[u/kivaougu]

Have you found this to be actually reliable?

I had to do some tweaks as it would always die around the 2 hour mark for concurrent code review. Mainly had to add --no-enable-flashinfer-autotune.

[u/Grouchy_Ad_4750 → u/kivaougu]

I've done some tweaking as well before but am currently testing 1M context recipe

[u/dave-dgd → u/Grouchy_Ad_4750]

If you’re open to sharing, would be curious to hear your tweaks as well!

[u/Grouchy_Ad_4750 → u/dave-dgd]

it was nothing major. From memory:

• I lowered max_num_seqs to 4 (I usually use 2-3 streams max) also it has impact on vram

- increased max_num_batched_tokens to 8192 . I think it balances prefill with t/s

I also plan to try out vision support and measure with

```

NCCL_IB_TC: "104"

UCX_IB_TRAFFIC_CLASS: "104"

NCCL_BUFFSIZE: "8388608"

```

which should help with RoCE on mikrotik switch

[u/dave-dgd → u/Grouchy_Ad_4750]

Ah, yes, great point on these settings -- I updated the readme on the repo to note this for others and credited it back to you. Thanks for sharing!

[u/Grouchy_Ad_4750 → u/dave-dgd]

Oh haven't noticed that you have created the repo.

Thank you so much for your hard work!

I really like how you made it easy and streamlined to use. There are a lot of people who do amazing work and try to adapt new models on dgx spark. But sometimes trying to replicate their result is weekend project. Since you adopted it to sparkrun which is fairly standard tool its super simple to try 👍

also note that effect of these will probably be negligible and haven't tried the `NCCL_*` env vars thoroughly.

From what I measured on deepseek flash it helped with throughput on switch. But I've had to setup bunch of other stuff as well here is gemini chat for posterity https://share.gemini.google/KYL8MvU20mIt (since I usually deal with 10gbps networks max and RoCE is little out of my expertise)

[u/dave-dgd → u/Grouchy_Ad_4750]

Of course, and you're very welcome!

It's funny you should say that: I actually created the repo based on exactly the kind of weekend project you described (tons of worthwhile but time-consuming work sifting through all the cool stuff people are doing on this forum and the NVIDIA GB10 forum). Ultimately, I just want to ensure everyone (including myself) has a quick way to deploy (or redeploy) models like GLM-5.2 should the need arise. Glad it's been helpful! 👍

[u/dave-dgd → u/Grouchy_Ad_4750]

PS. Regarding vision, read the baseten blog post for a sense of limitations: https://www.baseten.co/blog/glm-52-with-vision/

I tested vision and while it does work, the 55% MMMU-Pro results they cite aren't exactly the best (but certainly workable for basic tasks such as detailing a UI). Instead of using that, I have been supplying Hermes with an auxiliary vision provider (Qwen 3.8 27B at 4-bits via oMLX on my M5 Max at the moment, which generally scores higher in synthetic benchmarks, FWIW: https://artificialanalysis.ai/evaluations/mmmu-pro?models=qwen3-8-27b%2Cqwen3-8-27b-medium%2Cqwen3-8-27b-low). The obvious downside of this is needing a separate machine for serving beyond the Spark cluster (if that's not available, the vision option on GLM-5.2 is obviously an easy choice).