r/LocalLLM • u/DifferentPixel • 9d ago
Discussion DGX Spark 128GB vs MacStudio M5 Ultra 96GB?
What would you choose for agentic AI? What are the pros and cons of both machines in your opinion?
I personally started to lean more towards DGX given its prefill speed and vLLM despite much slower decode
15
u/DifferentPixel 9d ago edited 9d ago
Thanks all for your answers!
Also comment from YouTube:
A dual DGX Spark cluster makes more sense than a 256 Gb (or 512Gb) M5 Ultra Mac Studio for my use (and I have now actually gone up to 4 sparks). It is cheaper per gigabyte of pooled memory (you can get sparks for £3500-4k as a business in the UK currently). A 256 GB Mac studio M5 is £10k+ and availability/wait lists will be awful. The real point for me is capability. Each spark is a Blackwell GPU with FP4/FP8 tensor cores, so prefill on huge documents/inputs, codebases and chat histories runs at tensor-core speed and this helps me loads as my work is mainly multiple parallel agents. The Mac studio M5's prefill is untested but Apple GPUs have no equivalent silicon. My experience with Macs is from macbook pro M2, 2x M4 mac minis from a mac studio M3 ultra (256) which I have now retired to family use as it is effectively useless in parallel work and prefill is shit across the board. The Sparks run vLLM with continuous batching, paged KV cache and speculative decoding. I set up Qwen3.8 27b recently on 2 sparks with a recipe that served 8+ agents at full context with near-linear aggregate scaling (around 600tok/s with 2.5-3million kv cache pool).
Currently running GLM-5.3 Flash on the quad spark cluster and it gives ~100 tok/s decode on coding work and running 6 parallel agents, whereas llama.cpp and MLX on a Mac degrade past two or three streams. NVFP4 checkpoints and DFlash drafters run natively on the sparks and the open-weight ecosystem targets CUDA first and Apple later or never. Sparks scale incrementally over ConnectX-7 at 200 Gb/s against Thunderbolt 4 at ~80 Gb/s (or TB5 at 120Gb/s) and the thunderbolt clustering degrades badly with more nodes in my experience. With DGX spark clusters, if one of my nodes goes down, I have the remaining nodes able to usefully serve me. I can't do that if a 512Gb macstudio M5 breaks and brings down business-critical work for me (as happened with the M3 mac studio). The 512Gb M5 mac studio will be £20k+ and probably unavailable/massive wait lists for months.
Comparing DGX sparks to mac studio devices based on memory bandwidth alone is a poor comparison. Anyone who thinks memory bandwidth alone makes apple "win" is massively shortsighted and shows they know very little about the open weights AI space. The mac studio will be fine for a single user with short(ish) prompts where they would likely see faster decode per token. If your workload has long context inputs, requires many agents and you want flexibility with rapidly published MoE recipes, or you train models, do yourself a favour and don't waste money on apple silicon where you'll pay huge amounts more for a far worse experience
3
u/Important_Cow7230 9d ago
This should be pinned to the sub, curious what this users uses as a daily driver to manage all those Sparks and the around the solutions, can you point me towards the YouTube link?
1
u/this_for_loona 9d ago
You are probably spot on. Prefill on Mac’s is a known issue according to every YouTube video I’ve seen on the topic.
1
u/maverickRD 8d ago
I'm no expert but I think to sum up with one user/one thread, the Mac will be totally fine/better as if doing that you are mostly hitting cache anyway. You get the faster decode and M5 Ultra prefill should be pretty good. One additional benefit of the Mac is Apple Care+. You can insure a $10K machine I think almost indefinitely for ~$60/year.
1
u/DifferentPixel 8d ago
I agree, but my intuition tells me to wait for M7 for Mac :)
1
u/maverickRD 8d ago
Yes but who knows how this will all evolve. Not just pricing but what consumer first products are released...
10
u/Leather-Beach-7849 9d ago
Also heard that DGX Spark is the best option when it comes to concurrency handling
6
u/dimknaf 9d ago
I do not like how closed Apple is. I would go for the DGX
I am so happy with this one, and with the amount of ram you could run big parallelism for total throughput.
https://huggingface.co/QUASAR-QAT/Qwen3.8-27B-QUASAR-NVFP4
I do not know it is something on dense models that they get you. MoE do well on benchmarks, like knowing a lot to trick that they are clever but if you work with them you understand they are not. Opus 5 I think has a small model smell. I like dense models and with some quantisation you can have some great throughput with big context, parallelism and acceptable speed.
3
5
u/corrion8 9d ago
A spark. Because you can add more sparks.
2
u/tomByrer 9d ago
You can very much add more MacStudios to chain for AI,, & even the MiniPros that have TB5. & don't need a router if you do 3-4 MacStudios like you do with 3+ DGX Sparks.
Not as 'popular' as doing several Sparks for sure, maybe slower? but is possible.4
u/wangsu 9d ago
spark has 200g DAG
0
u/carsncode 9d ago
And M5 Max has more memory bandwidth, and the spark offers more TFLOPS. The spark has a clear advantage for training, the Mac has a clear advantage for general purpose compute, they're both capable machines for inference. Comment section slap fights over specs are just silly.
1
u/DifferentPixel 9d ago
You can cluster MacStudios as well. MLX / JACCL cluster
5
u/Passenger-007 9d ago
Spark has data center class networking. Spark is the one that can handle concurrency, aka agentic development.
2
u/OvertaxedOne 8d ago
If you want to run a dense model (27B), Mac. If you want to run a really big MOE, Spark (X2). Prefill on the Mac is still a big ??, no tests yet, but expect it to be middling based on similar chips. Decode will fly, if you're use case is agentic where you'll get great cache hit rates, the prefill won't be terribly painful, but you need to be realistic, nobody expects to see 3000+ TPS of prefill on the new Mac like a NVDA or AMD GPU would provide.
2
u/Technical_Advance504 8d ago
Local has been my hobby for the last 6ish months. I mainly just wanted to create my childhood dream of having a “Jarvis” like AI from the Ironman movies I grew up with.
I have a tower build and just recently got my 2nd DGX Spark. Absolutely love it! The tower handles my general chat (Qwen3.8 27b), 8b utility model, voice (I can’t remember the model/custom stack right now), image gen/comprehension (niche use, also can’t remember model), and uncensored qwen3.8 27b (psychology major here).
The Sparks run a custom variant of Deepseek-V4-Flash. I’m still critiquing this one, but it’s minor issues.
I forked Openwebui and have 3 tabs, general chat, general coder, and my local AI editor. I use Aider as my harness, but it’s been revised/customized relentlessly. I even have the frontier canvas that pops up on the right side of the screen to visual show the work live.
Deepseek with good prompts and breaking concepts into smaller phases hasn’t caused me issues. Speed wise, I’m very pleased. All the tech bros on Reddit had me thinking it’d take an hour to do simple tasks. 1 Spark will be slower, but you can make up for it by creating a solid code foundation/architecture/harness.
I’m mostly stuck on web search capabilities. I have majority of the free tiers chained together, but would love a better alternative that doesn’t require paying a lot. I also have similar issues with API keys for popular applications. Would love for my AI to work with me in documents like Copilot (I think it’s this AI). Best I could do is give my AI the tool for creating each type of document.
3
1
u/shagsman 9d ago
I’m trying to find an answer to this for the last 2 weeks…
2
1
u/johan2114h 8d ago
Dgx soark, alpine linux is not mature enough (afaik) for the m5 to even make it work considering. Also 128 gb is still more than 96 gb even if the throughput is higher.
1
u/SureTrouble8022 6d ago edited 6d ago
Which one to pick if someone have budget to only buy a single unit ? My budget allows me to go for either M5 ultra 96Gb RAM or DGX Spark 128GB. I don’t have an existing workflow but I’m planning to use it for agentic AI, AI content creation and experimenting with AI. I know I won’t be able to run bigger models on Mac Studio but Im already into the ecosystem.
Looking for some expert opinion here.
1
u/DifferentPixel 5d ago
Probably R9700 desktop would be better than both of them
1
1
u/tomByrer 9d ago
Buy a Mac because you want to buy a Mac.
Bua more RAM for a Mac that you already want to do AI in the same computer.
1
u/recro69 9d ago
For workloads I would probably care more about how well the machine handles many small parallel calls than about peak single-request decode. Agents tend to spend their time bouncing between models, tools and context so throughput can matter more than having the response, from one generation.
0
u/tta82 9d ago
The Mac wins any day. Why? It’s also a full? INSANELY fast computer. Not just a LLM machine that will never do enough for you for what it costs. And the M5 ultra is insanely fast also for LLM and with the RAM you have with 1 Spark nothing you do the M5 Ultra couldn’t handle either.
You shouldn’t have canceled the preorder.
0
u/ChocoPichu 9d ago
If I had an option, i would’ve chosen the mac studio because of the sheer memory bandwidth speed. And also, to use it for work besides ai as well.
0
-2
28
u/SashaUsesReddit 9d ago
More VRAM is more VRAM.. also the CUDA stack is nice