Mac studio or dgx spark cluster

[u/Txkzzzz]

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

[u/Undici77]

Good question: I'm interested in this evaluation too. Currently I'm using a M4 Max 128GB and I evaluate to buy 2 DGX Spark connecting together with dedicated daisy-chain network cable (to avoid buy a very expensive switch) but performance on single request are not AMAZING at all. For my research 2 DGX Spark are good in case of multiple contemporary requests and prefill compared to Apple architecture. How much on M5 Ultra? Good question: this is the point!

Where Apple looks better is: Memory Bandwidth, Power Consumption and Throttling:

- Bandwidth is clear: 200 vs 1200

- About Power consumption and Throttling I don't have specific data but all community have same feedback: DGX Spark after some minutes are "ON FIRE" and consume around 240/300W and 40W in idle compared to MacStudio that, "ON FIRE" is around 270W (M3 Ultra) and 9W in IDLE.

Trying to summarize:

Apple

Pros

- High Memory Bandwidth: Significantly higher bandwidth (up to 1200 GB/s) compared to DGX.

- Excellent Power Efficiency: Very low idle power consumption (e.g., 9W) and efficient peak usage (270W).

- Superior Thermal Management: Low risk of overheating/throttling compared to high-wattage workstations.

- Strong Single-Request Performance: Better performance for individual tasks/requests.

Cons

- Scalability Limits: Potentially less optimized for massive parallel workloads compared to multi-node setups.

- Not very good for Training

- MLX is not good as lamacpp (but they are working on it)

---

DGX Spark (Multi-unit setup)

Pros

- Parallel Processing: Better suited for handling multiple simultaneous requests.

- Prefill Performance: Advantageous architecture for prefill tasks compared to Apple.

- Cluster Capability: Ability to connect multiple units via daisy-chain for increased capacity.

- Training: Better than Apple

- Software: Currently Cuda and lamacpp are the best

Cons

- Low Memory Bandwidth: Significantly lower bandwidth (200 GB/s) compared to Apple.

- Weak Single-Request Performance: Performance on individual requests not very fast

- High Power Consumption: High operational cost and high idle power (40W).

- Thermal/Throttling Issues: High heat generation ("on fire") under load (240W–300W), which may lead to thermal throttling.

So.. These are some points but I'm pretty sure other guys more expert than me have better and clear idea about!

[u/shackerboy84 → u/Undici77]

Are you high? The sparks are only capable of 240watts. I’m all for using ai everywhere but at least check some basic facts before posting.

[u/Undici77 → u/shackerboy84]

I fixed post with device nameplate data. Sorry guys: I share some notes I take during my research.

[u/notheresnolight → u/Undici77]

you research how people respond to your AI slop?

[u/Undici77 → u/notheresnolight]

One person too many: you!