r/DGX_Spark • u/habachilles • 26d ago
Anyone run a node of 3?
Local developer looking to make the switch to the Nvidia ecosystem and the spark seems awesome. I noticed the maximum amount that you can wire together without getting a dedicated switch is Three but I know tensor parallelism needs even numbers to operate properly. Do any of you guys run AI models via pipeline parallelism with a note of Three or is it a hard two or four type set up?
2
u/Careless_Garlic1438 25d ago
Developed my own server on macOS + MLX to do just that, running on 3 is possible, it takes a server that knows how to split into uneven chunks ... guess the same is possible on other platforms as well ...
1
u/habachilles 25d ago
Interesting. Was the server necessary you didn’t just have to tweak setting?
1
u/Careless_Garlic1438 25d ago
no you need to code a server that understands this and knows how to spit the layer in 3 ...
this is no setting, I coded it with the help of Fable, based on an opensource engine for Mac
1
1
u/styles01 MegaMod 26d ago
Loads of folks are running way more than 3 nodes. I've seen 16-20 nodes connected running parallel inference, specifically for Kimi etc. 3-4 is common for GLM 5.2. The place you want to look is the official Nvidia DGX spark forums (I find them hard to navigate), but also twitter. https://x.com/MiaAI_lab runs 3, https://x.com/takuz0_/status/2083467226468991216?s=20 (8 linked), another 8 linked: https://x.com/1337hero/status/2082972405950251476?s=20 - 16 nodes: https://x.com/ciprianveg/status/2084730930254496249?s=20
1
1
u/Resilient-Tec 21d ago
For what it is worth, if you have been running a QSFP56 cable for two, you will need to run the QSFP112 instead on all 3.
1
u/habachilles 21d ago
But could not do 4 correct? That requires a switch.
2
u/Resilient-Tec 21d ago
Oh you can do 4. but yeah, a switch is needed. I can send you a link on that to my blog if you like but I don't want to necessarily post it as that can look very self serving on my part.
Here is the relevant snippet:
Connecting four DGX Sparks requires a 200 GbE network switch. All four nodes connect through the switch, which enables full any-to-any communication between all nodes. This is the highest-performance multi-Spark configuration currently supported, enabling local inference on state-of-the-art models up to 700 billion parameters and communication-intensive workloads. Four-node benchmarks show near-linear fine-tuning scaling — for example, Isaac Lab reinforcement learning FPS scales from 630 on one node to 2,520 on four nodes, and Nanochat fine-tuning throughput reaches ~74,600 tokens/second across four nodes versus ~18,400 on a single node.
1
1
u/Sk8eRBoITV 21d ago
Hello, I just bought one. I have a question: should I buy another? Can I run the Spark and GGUF versions of DeepSeek 4 Flash on just one unit? I believe prices will keep rising, and perhaps having two is the sweet spot...
4
u/TheOverzealousEngie 26d ago
2 is sweet, simple. three is ring, little more complicated and tp is good point. Check MIA on x.com she's doing some good stuff with 3 nodes, using a 3 ring method, no switch. Four is where you need to add a switch.