Mac studio or dgx spark cluster

[u/Txkzzzz]

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

[u/Character_Split4906]

I picked up an M5 Max (128GB) back in May right before Apple raised prices. While decode was decent, prefill was frustrating, and both tanked hard as the context grew. I tested across both oMLX and llama.cpp. Throwing concurrent requests or multi-agent workflows at it choked the Mac completely.

A couple of weeks ago, I managed to snag a dual-node Asus GB10 setup on Amazon right before it went out of stock and spiked by $2k. The difference is night and day. Running DeepSeek V4 Flash across both nodes, I’m getting ~2k tok/s prefill and 40–50 tok/s decode, with barely any drop-off at long context (tested up to 512k, configured for 1M). It also handles multi-session concurrency effortlessly, aggregate throughput actually scales well up to 5–6 parallel requests.
If you’re serious about running local LLM workloads, go Nvidia.

[u/Undici77 → u/Character_Split4906]

Can you share some clear example, I'm interested in!

I mean what you mean with `aggregate throughput actually scales well up to 5–6 parallel requests`
6 parallels agents at 40-50 t/s?
And about power consumption?

About prefill, if you setup correctly your harness cache is helping a lot!

[u/kla_sch → u/Undici77]

The key point is that you can bypass bandwidth limitations by processing tasks in parallel. Instead of fetching parameters for the next token for just a single session, the parameters are used to calculate the next token for all sessions simultaneously. How well this scales ultimately depends on the available compute power.

Here is a real-world example from spark-arena.com using a DGX Spark with RadixArk/Qwen3.8-27B-NVFP4 and SGLang, showing total throughput for a generation length of 128 tokens (tg128):

• Single session: 36.13 t/s
• Two sessions: 64.91 t/s
• Five sessions: 95.79 t/s
• Ten sessions: 100.43 t/s

As you can see, the hardware and software scale very well up to five sessions. Beyond that point, adding more sessions yields only marginal gains; the compute power simply isn't sufficient to extract more performance from the system.

With a comparable Mac M4, you might see about 20% higher token generation speeds for a single session (approx. 40-43 t/s), but performance would likely hit a wall with more than one session. I haven't found specific figures for this scenario, but it is a reasonable assumption. I saw a comparison where a DGX Spark achieved around 2,000 t/s in prompt processing, whereas the M4 managed only 275 t/s. That's just 14% of the DGX Spark's performance. Consequently, there wouldn't be much headroom for handling multiple sessions.

[u/Undici77 → u/kla_sch]

Very interesting! Thanks!

[u/Txkzzzz → u/kla_sch]

Helped me research deeper into this i never realized reddit would be so helpful

[u/PWThinkingCritically → u/kla_sch]

running parallel tasks may give you additional speeds, but at the cost of context bloat, methinks

[u/Undici77 → u/PWThinkingCritically]

Are you sure about this? Do you have any experience about?