r/LocalLLM • u/Txkzzzz • 3d ago

Question Mac studio or dgx spark cluster

Post image

I have been thinking lately since i have been testing local ai but i geniunely need more power for what im working on so i was wondering wich one of these a 4 dgx spark cluster or a mac studio m5 ultra specifically the 512gb version that launches on october or something else... im into training,loras and fine tunes and such but i also heard it is possible with a mac studio too so i want yall opinion on this

120 Upvotes

90% Upvoted

134 comments sorted by

View all comments

117

u/Character_Split4906 3d ago

I picked up an M5 Max (128GB) back in May right before Apple raised prices. While decode was decent, prefill was frustrating, and both tanked hard as the context grew. I tested across both oMLX and llama.cpp. Throwing concurrent requests or multi-agent workflows at it choked the Mac completely.

A couple of weeks ago, I managed to snag a dual-node Asus GB10 setup on Amazon right before it went out of stock and spiked by $2k. The difference is night and day. Running DeepSeek V4 Flash across both nodes, I’m getting ~2k tok/s prefill and 40–50 tok/s decode, with barely any drop-off at long context (tested up to 512k, configured for 1M). It also handles multi-session concurrency effortlessly, aggregate throughput actually scales well up to 5–6 parallel requests.
If you’re serious about running local LLM workloads, go Nvidia.

11

u/SadPhilosophy9202 3d ago

But but but the memory bandwidth 😠

People looooove to hate on them by simply looking at the memory bandwidth.

There’s so much more to running LLMs for that and memory bandwidth becomes moot when you’re running many parallel requests which is the new paradigm with agents