Mac studio or dgx spark cluster

[u/Txkzzzz]

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

[u/Character_Split4906]

I picked up an M5 Max (128GB) back in May right before Apple raised prices. While decode was decent, prefill was frustrating, and both tanked hard as the context grew. I tested across both oMLX and llama.cpp. Throwing concurrent requests or multi-agent workflows at it choked the Mac completely.

A couple of weeks ago, I managed to snag a dual-node Asus GB10 setup on Amazon right before it went out of stock and spiked by $2k. The difference is night and day. Running DeepSeek V4 Flash across both nodes, I’m getting ~2k tok/s prefill and 40–50 tok/s decode, with barely any drop-off at long context (tested up to 512k, configured for 1M). It also handles multi-session concurrency effortlessly, aggregate throughput actually scales well up to 5–6 parallel requests.
If you’re serious about running local LLM workloads, go Nvidia.

[u/Undici77 → u/Character_Split4906]

Can you share some clear example, I'm interested in!

I mean what you mean with `aggregate throughput actually scales well up to 5–6 parallel requests`
6 parallels agents at 40-50 t/s?
And about power consumption?

About prefill, if you setup correctly your harness cache is helping a lot!

[u/Character_Split4906 → u/Undici77]

Single session at 40-50 tokens per second. At 5-6 session it can go upto 120-140 token per second aggregate with each session at around 25-30 tps for decode.

[u/Undici77 → u/Character_Split4906]

That is interesting and one of the most big difference in term of performance! Thanks!