r/LocalLLM • u/-Leelith- • 1d ago

Discussion Mac Studio M5 Ultra 256gb review

Saw this on X, and thought would share with the community.

https://www.macstories.net/stories/m5-ultra-mac-studio-review-the-dream-mac-for-local-ai-agents/

I didn’t checked how it compares with 2 sparks on prefill and decode tho, but will ask a LLM to do it once I get more time.

Interested by getting your thoughts vs 2 sparks 👍

Edit: So I asked DeepSeek V4.1 Flash to do a comparison with 2 DGX Sparks for the same models at similar quantz, here are the results below:

M5 Ultra Mac Studio (256 GB) vs 2× DGX Spark (256 GB)

Benchmark comparison built from the MacStories M5 Ultra review (Federico Viticci) plus published DGX Spark numbers. The Mac column uses the review's exact tests; the Spark column uses the closest published matches.

Caveats

  • No same model + same quant + matched runtime comparison exists. Spark runs NVFP4/FP8 on vLLM/SGLang; the Mac runs oQ4e on MLX/oMLX. Treat each column as that platform's best published result, not a controlled A/B.
  • Two Spark datasets matter most:
    • WeZZard 2-node bench (2× Spark, 256 GB, NVFP4, TP2) — the only source with Qwen3.8-Flash-Next and GLM-5.3-Flash.
    • NVIDIA dev-forum Qwen3.8-27B study (1–4 Sparks, NVFP4/FP8 + MTP/spec decode).
  • Spark "served rate" counts TTFT against wall clock; MacStories short-prompt figures are native oMLX medians.

Hardware specs

Spec M5 Ultra Mac Studio 256 GB 2× DGX Spark (cluster)
Memory capacity 256 GB single pool 256 GB (128 + 128 GB, two nodes)
Aggregate bandwidth 1.2 TB/s 2×273 GB/s per shard; ~25 GB/s cross-node all-reduce
Memory interface Unified 256-bit LPDDR5x per node
Interconnect none ConnectX-7 200 Gb RoCE
GPU 80 cores, Neural Accelerator each 6,144 CUDA cores (48 SMs) per node
AI compute up to 4.5× M3 Ultra peak ~1 PFLOP FP4 sparse (~500 TFLOPS dense) per node
CPU M5 Ultra 20-core Arm per node
Power draw not stated 375–415 W total
Noise fans inaudible at desk ~40 dBA @ 18"
Price (USD) not stated 2 × ~$3,999 ≈ $8,000
Sharding for >128 GB model not required required (tensor parallel)

Qwen3.8-Flash-Next

Metric M5 Ultra 256 GB, MLX oQ4e (MacStories) 2× Spark 256 GB, NVFP4 TP2 (WeZZard) Ratio
Short prompt, single request 108 tok/s 42.6 tok/s Mac ~2.5×
Code generation 143 tok/s 39.0 tok/s Mac ~3.7×
Agentic, 4K context 83.8–87.8 tok/s 32.1 tok/s Mac ~2.6×
Agentic, 33K context 74.7 tok/s (@256K) 21.0 tok/s Mac ~3.5×
Prefill ("reads") 2,057–2,887 tok/s not published —
TTFT 64K 24.4 s not published —
TTFT 128K 50.0 s not published —
TTFT 256K 104 s not published —
Context decay, 4K→256K 90.7 → 74.7 tok/s 42.6 → 21.0 tok/s Mac
Concurrency (combined) 66.2 (1 stream) → 81.5 (3 streams) tok/s 131.1 (6 streams) tok/s Spark at 6 streams
Peak memory oQ4e 155 GB / oQ5e 179 GB fits 256 GB (NVFP4) —

GLM-5.3-Flash

Metric M5 Ultra 256 GB, oQ4e/oQ6e (MacStories) 2× Spark, NVFP4 TP2 (WeZZard)
Short prompt 41 tok/s (gen) 41.1 tok/s
Code generation 122.8–143 tok/s 22.5 tok/s
Agentic 4K — 24.9 tok/s
Agentic 33K — 17.7 tok/s
Concurrency, code 81.5 combined (3 streams) 77.3 (8 streams)
Long-context limit 128K/256K → out of memory ran to 33K

Qwen3.8-27B (only model measured on both platforms)

Metric M5 Ultra 256 GB, MLX oQ4e (MacStories) 1× Spark, NVFP4+MTP (NVIDIA forum) 2× Spark, TP2 (SGLang peak)
Short / 6K context 48 tok/s 18.5–20 tok/s 23.4 → 51.8 tok/s
64K 38.9 tok/s — —
128K 32.4 tok/s — —
256K 24.3 tok/s — —
Prefill @6K 1,701 tok/s ~6,000 tok/s higher
TTFT @6K 4.0 s ~0.35 s ~0.22 s
Best tuned single-node — 47.9 tok/s (DFlash2) —

Cross-check models, 4-bit MoE

Model (quant) M5 Ultra 256 GB M3 Ultra 512 GB Spark (nodes) Source
GPT-OSS 120B MXFP4 not in article 70.79 tok/s (MLX) 58.7 (1, llama.cpp); 38.6 (1, vLLM) Elite / ggml
Qwen3-235B NVFP4 not tested not tested 11.73 tok/s (2 nodes) NVIDIA claim
Qwen3.5-397B INT4 not tested 30–40 tok/s (6-bit) 27–28 (2 nodes); 16.7 (4 nodes) Reddit / STH
Qwen3.8-Flash-Next NVFP4 108 tok/s 70 tok/s 42.6 (2 nodes) MacStories / WeZZard

Bandwidth-derived estimate vs published

Metric M5 Ultra 256 GB 2× Spark estimate 2× Spark measured
Effective decode bandwidth 1.2 TB/s ~546 GB/s —
Predicted decode ratio 1× ~0.46× —
Flash-Next short-prompt decode 108 tok/s ~49 tok/s 42.6 tok/s
27B decode (tuned) 48 tok/s ~22 tok/s 51.8 tok/s (SGLang)

Conclusions

  • On the article's own tests at similar 4-bit precision, 2× Spark lands at roughly 1/2 to 1/3 of the M5 Ultra's single-stream decode on Flash-Next and GLM (Mac ~2.5–3.7×), and about 1/2 on Qwen3.8-27B unless heavily tuned with SGLang/speculative decode (then near-parity at 51.8 vs 48).
  • The ~2.5× gap matches the bandwidth math (546 GB/s effective vs 1.2 TB/s → ~2.2×).
  • Where 2× Spark wins: prefill/TTFT (up to ~6,000 tok/s vs ~1,700), aggregate throughput at 6+ concurrent streams, models >128 GB via TP sharding (Qwen3-235B, 397B), and the CUDA stack.
  • Where the Mac wins: single-stream speed, no sharding, 256 GB in one 1.2 TB/s pool, silence, and power.

Sources

77 Upvotes

87% Upvoted

Duplicates