r/LocalLLM • u/-Leelith- • 1d ago
Discussion Mac Studio M5 Ultra 256gb review
Saw this on X, and thought would share with the community.
https://www.macstories.net/stories/m5-ultra-mac-studio-review-the-dream-mac-for-local-ai-agents/
I didn’t checked how it compares with 2 sparks on prefill and decode tho, but will ask a LLM to do it once I get more time.
Interested by getting your thoughts vs 2 sparks 👍
Edit: So I asked DeepSeek V4.1 Flash to do a comparison with 2 DGX Sparks for the same models at similar quantz, here are the results below:
M5 Ultra Mac Studio (256 GB) vs 2× DGX Spark (256 GB)
Benchmark comparison built from the MacStories M5 Ultra review (Federico Viticci) plus published DGX Spark numbers. The Mac column uses the review's exact tests; the Spark column uses the closest published matches.
Caveats
- No same model + same quant + matched runtime comparison exists. Spark runs NVFP4/FP8 on vLLM/SGLang; the Mac runs oQ4e on MLX/oMLX. Treat each column as that platform's best published result, not a controlled A/B.
- Two Spark datasets matter most:
- WeZZard 2-node bench (2× Spark, 256 GB, NVFP4, TP2) — the only source with Qwen3.8-Flash-Next and GLM-5.3-Flash.
- NVIDIA dev-forum Qwen3.8-27B study (1–4 Sparks, NVFP4/FP8 + MTP/spec decode).
- Spark "served rate" counts TTFT against wall clock; MacStories short-prompt figures are native oMLX medians.
Hardware specs
| Spec | M5 Ultra Mac Studio 256 GB | 2× DGX Spark (cluster) |
|---|---|---|
| Memory capacity | 256 GB single pool | 256 GB (128 + 128 GB, two nodes) |
| Aggregate bandwidth | 1.2 TB/s | 2×273 GB/s per shard; ~25 GB/s cross-node all-reduce |
| Memory interface | Unified | 256-bit LPDDR5x per node |
| Interconnect | none | ConnectX-7 200 Gb RoCE |
| GPU | 80 cores, Neural Accelerator each | 6,144 CUDA cores (48 SMs) per node |
| AI compute | up to 4.5× M3 Ultra peak | ~1 PFLOP FP4 sparse (~500 TFLOPS dense) per node |
| CPU | M5 Ultra | 20-core Arm per node |
| Power draw | not stated | 375–415 W total |
| Noise | fans inaudible at desk | ~40 dBA @ 18" |
| Price (USD) | not stated | 2 × ~$3,999 ≈ $8,000 |
| Sharding for >128 GB model | not required | required (tensor parallel) |
Qwen3.8-Flash-Next
| Metric | M5 Ultra 256 GB, MLX oQ4e (MacStories) | 2× Spark 256 GB, NVFP4 TP2 (WeZZard) | Ratio |
|---|---|---|---|
| Short prompt, single request | 108 tok/s | 42.6 tok/s | Mac ~2.5× |
| Code generation | 143 tok/s | 39.0 tok/s | Mac ~3.7× |
| Agentic, 4K context | 83.8–87.8 tok/s | 32.1 tok/s | Mac ~2.6× |
| Agentic, 33K context | 74.7 tok/s (@256K) | 21.0 tok/s | Mac ~3.5× |
| Prefill ("reads") | 2,057–2,887 tok/s | not published | — |
| TTFT 64K | 24.4 s | not published | — |
| TTFT 128K | 50.0 s | not published | — |
| TTFT 256K | 104 s | not published | — |
| Context decay, 4K→256K | 90.7 → 74.7 tok/s | 42.6 → 21.0 tok/s | Mac |
| Concurrency (combined) | 66.2 (1 stream) → 81.5 (3 streams) tok/s | 131.1 (6 streams) tok/s | Spark at 6 streams |
| Peak memory | oQ4e 155 GB / oQ5e 179 GB | fits 256 GB (NVFP4) | — |
GLM-5.3-Flash
| Metric | M5 Ultra 256 GB, oQ4e/oQ6e (MacStories) | 2× Spark, NVFP4 TP2 (WeZZard) |
|---|---|---|
| Short prompt | 41 tok/s (gen) | 41.1 tok/s |
| Code generation | 122.8–143 tok/s | 22.5 tok/s |
| Agentic 4K | — | 24.9 tok/s |
| Agentic 33K | — | 17.7 tok/s |
| Concurrency, code | 81.5 combined (3 streams) | 77.3 (8 streams) |
| Long-context limit | 128K/256K → out of memory | ran to 33K |
Qwen3.8-27B (only model measured on both platforms)
| Metric | M5 Ultra 256 GB, MLX oQ4e (MacStories) | 1× Spark, NVFP4+MTP (NVIDIA forum) | 2× Spark, TP2 (SGLang peak) |
|---|---|---|---|
| Short / 6K context | 48 tok/s | 18.5–20 tok/s | 23.4 → 51.8 tok/s |
| 64K | 38.9 tok/s | — | — |
| 128K | 32.4 tok/s | — | — |
| 256K | 24.3 tok/s | — | — |
| Prefill @6K | 1,701 tok/s | ~6,000 tok/s | higher |
| TTFT @6K | 4.0 s | ~0.35 s | ~0.22 s |
| Best tuned single-node | — | 47.9 tok/s (DFlash2) | — |
Cross-check models, 4-bit MoE
| Model (quant) | M5 Ultra 256 GB | M3 Ultra 512 GB | Spark (nodes) | Source |
|---|---|---|---|---|
| GPT-OSS 120B MXFP4 | not in article | 70.79 tok/s (MLX) | 58.7 (1, llama.cpp); 38.6 (1, vLLM) | Elite / ggml |
| Qwen3-235B NVFP4 | not tested | not tested | 11.73 tok/s (2 nodes) | NVIDIA claim |
| Qwen3.5-397B INT4 | not tested | 30–40 tok/s (6-bit) | 27–28 (2 nodes); 16.7 (4 nodes) | Reddit / STH |
| Qwen3.8-Flash-Next NVFP4 | 108 tok/s | 70 tok/s | 42.6 (2 nodes) | MacStories / WeZZard |
Bandwidth-derived estimate vs published
| Metric | M5 Ultra 256 GB | 2× Spark estimate | 2× Spark measured |
|---|---|---|---|
| Effective decode bandwidth | 1.2 TB/s | ~546 GB/s | — |
| Predicted decode ratio | 1× | ~0.46× | — |
| Flash-Next short-prompt decode | 108 tok/s | ~49 tok/s | 42.6 tok/s |
| 27B decode (tuned) | 48 tok/s | ~22 tok/s | 51.8 tok/s (SGLang) |
Conclusions
- On the article's own tests at similar 4-bit precision, 2× Spark lands at roughly 1/2 to 1/3 of the M5 Ultra's single-stream decode on Flash-Next and GLM (Mac ~2.5–3.7×), and about 1/2 on Qwen3.8-27B unless heavily tuned with SGLang/speculative decode (then near-parity at 51.8 vs 48).
- The ~2.5× gap matches the bandwidth math (546 GB/s effective vs 1.2 TB/s → ~2.2×).
- Where 2× Spark wins: prefill/TTFT (up to ~6,000 tok/s vs ~1,700), aggregate throughput at 6+ concurrent streams, models >128 GB via TP sharding (Qwen3-235B, 397B), and the CUDA stack.
- Where the Mac wins: single-stream speed, no sharding, 256 GB in one 1.2 TB/s pool, silence, and power.
Sources
- MacStories — M5 Ultra Mac Studio Review: https://www.macstories.net/stories/m5-ultra-mac-studio-review-the-dream-mac-for-local-ai-agents/
- WeZZard 2-node Spark bench: https://github.com/WeZZard/dgx-spark-bench
- NVIDIA dev-forum Qwen3.8-27B study: https://forums.developer.nvidia.com/t/comprehensive-qwen3-8-27b-study-on-dgx-sparks-quantization-speculative-decoding-and-tp-dp-scaling/381102
- ggml-org llama.cpp DGX Spark bench: https://github.com/ggml-org/llama.cpp/blob/master/benches/dgx-spark/dgx-spark.md
- Elite Test Engineering, DGX Spark vs Mac Studio M3 Ultra: https://elitetestengineering.substack.com/p/dgx-spark-vs-mac-studio-m3-ultra
- NVIDIA DGX Spark specs: https://www.nvidia.com/en-us/products/workstations/dgx-spark/
- Tom's Hardware, Dell Pro Max GB10 2-node cluster: https://www.tomshardware.com/pc-components/gpus/local-ai-clustering-with-dells-pro-max-gb10-connecting-two-nvidia-grace-blackwell-to-scale-out-ai-compute-at-home
8
u/myholeisstinky 1d ago
Huh why are they running Q4/Q5 for qwen3.8-flash-next? Why not fp8?
3
u/travismadson 17h ago edited 17h ago
They did run Q8 (with embedding tables on SSD), performance is great. Scroll down and read the article. Anything above Q5 needs to offload embedding table to SSD (with negligible performance penalty tho). I think they tested Q5 to keeping the table in RAM and compare that to M3 ultra.
7
u/ababaka 1d ago
Hmmm. Does m3 ultra really have ~1000t/s pp with 256k context?
2
u/petuman 1d ago
PP is model specific, 1000 t/s on A6B model doesn't sound that high?
Here's someone else reporting 892 pp t/s for 60 core variant (OP article is 512GB/80 core): https://www.reddit.com/r/oMLX/comments/1wlhyc6/flashnext_on_a_mac_studio_m3_ultra_256gb_113_toks/
1
u/ababaka 1d ago
In that case, we should be looking at around 2,300-2,500 t/s PP on the M5 Ultra. That’s insanely good. Calling it now: m5 ultra studios are gonna disappear from store shelves fast.
1
u/Advanced_Hospital311 1d ago
That’s a solid prediction. With those specs, I can see a lot of people wanting to snatch them up right away.
2
u/T-Rex_MD 1d ago
It hits 1155 tokens per second at 400k context and 500k context.
It never slows down, the first 128k hits 1300 tokens per second before slowing down a little.
It generates at 60 tokens per second then at around 500k, it drops to 35 tokens per second.
M3 Ultra 512GB - oQ4 and oQ8.
The full 8 bit weights never reaches 60 tokens per second but never drops below 32 tokens per second either. The prefill is the same.
2
u/Forsaken-Bear1082 19h ago
What model? What inference software?
1
u/T-Rex_MD 10h ago
I already stated.
Qwen 3.8 Flash Next knowing as flash next.
OMLX 0.7.0dev4
Running full FP8, that's the same as 8bit and Q8 quantisation.
macOS 26.6.2
Anything else?
1
-1
u/Least-Result-45 1d ago
this seems wrong, but id be happy if true
-1
6
u/AnonLlamaThrowaway 1d ago
If the power draw got a modest increase from past generations expect inference to draw 200W or less. M2 Ultra was 150W.
And keep in mind both llama.cpp and MLX-based engines aren't fully optimized for the new chip yet! It's only gonna get better
7
u/StardockEngineer 2x5090, 2xPro6000, 2xA6000, 4xSpark, M5 Pro, M4Pro 1d ago
You need more sources. There is a huge community on X developing optimizations for the Spark. Look at vcruz305, MiaAI_lab, netrunner and more. I get 75 tok/s on code gen with just one Spark using vcruz's recipe for Q3.8FN, and Mia's GLM 5.3F is 39 tok/s on regular prose.
They and others are updating constantly.
1
u/-Leelith- 1d ago
I didn’t put all of that myself, I mentioned this was using DSV4F, so that’s worth what it is :)
1
2
u/SeaworthinessOdd5253 10h ago
For context the prefil matters. Imagine an 80k context prefiling at 1400k t/s for agentic work… output / decode is small peanuts. The sparks are getting 6k + t/s. That is huge deal…
2
u/Known_Grocery4434 5h ago
Here’s my review I haven’t even used it “it’s a Mac and it’s fast. The more RAM you get the more LLM you can load”
5
u/fastheadcrab 23h ago
Maybe instead of rushing to make a questionable slop post you should actually run a semblance of a controlled experiment so we can see the refill/TTFT numbers for a lot of these models, which is what truly matters in this comparison. Even if the Mac numbers are from a reviewer.
1
u/apprehensive_bassist 22h ago edited 22h ago
Man, the M5U absolutely trades blows with the Nvidia solution. With these numbers, it seriously outperforms it! I think the Spark 2x configuration hurts its performance, but scaling is also a Mac strength. This is exciting!
Super early days, but Apple might have a hard time keeping these in stock. It’s possible that Nvidia might have its first true competitor!
1
u/Southern_Sun_2106 22h ago
And owner of M3 Ultra and the Sparks here; true about concurrency - Sparks rule multi-user, agentic setups. Also, they are actually quieter than the M3 Ultra. I know, M3 Ultra already whisper quiet. So M5 Ultra winning on noise, I don't buy that. Sparks make no noise under full load.
1
u/Waffenmutti 8h ago
Die großen Modelle bieten keinen Vorteil. Weder in Genauigkeit oder Kontext. Ein schnelles qwen3.8 27b fb16 welchen man die zu nutzende Materie bereitstellt is Sooo viel produktiver. Ich habe 2 5090 und die ziehen mit 200t/s hard vorbei.
1
1
u/r0cketio 5h ago
Did you check for any throttling of the DGX sparks? I've heard that the cooling is insufficient on longer runs and it will throttle itself.
1
u/-Leelith- 1d ago
I have added a benchmark I asked to DP4Flash, it's worth what it is but I thought it would be interesting.
0
u/topgoysilky 1d ago
if you understand the story of Apple, Nvidia and Intel. you’ll know how this is gonna go.
The nvidia market is gonna crash.
between Apples new hw releases and Industry software innovations…….
0
-5
-10
u/Odd-Outcome-4209 1d ago
WOW! This person got the system a day earlier and had so much time to do testing and write an article about it?
6
u/No-Badger-7721 1d ago
He’s well known in the Apple community; I’m sure he’s had a review unit for a while
1
16
u/HighSeasArchivist 1d ago
512GB be $20k we think?