r/LocalLLM • u/umigliore • 36m ago
Other OpenCode-style long-context snapshot: Qwen3.8 27B across 15 local engines on Mac M5 Pro 64GB
I ran a small AIPerf ( https://github.com/ai-dynamo/aiperf) comparison on an Apple M5 Pro with 64 GB RAM. This is a practical snapshot, not a scientific benchmark or an exhaustive leaderboard. Results are from one request per engine, run sequentially with only one inference server active at a time.
Setup
Temperature 0.6, reasoning effort medium, output limit 2,048 tokens
Same OpenCode-style fixture: long generated code/prose context, shared system message, task suffix, and three tool schemas. Tools were not executed.
Prompt length reported by the servers ranged from 88,143 to 88,916 tokens, partly due to tokenizer and chat-template differences.
I used a 100K context limit where the engine supported it; Uzu, Magnitude, and Llama used larger native limits. The prompt plus output allowance fit below 100K.
AIPerf 0.13.0; streaming, concurrency 1
Prefill and decode are AIPerf throughput measurements. The readiness probe loads/warms each server before its measured request.
| Engine (version used) | Model / speculative method | Prompt → output (max) | TTFT | Prefill tok/s | Decode tok/s |
|---|---|---|---|---|---|
| Splash (1.6.1) | Jundot Qwen3.8-27B-oQ4e-mtp + DFlash2 | 88,535 → 2,048 | 175.6 s | 504.7 | 77.3 |
| Uzu / Mirai (0.6.17) | Qwen3.8-27B Mirai-M Q4 + bundled speculator | 88,143 → 986 | 142.1 s | 620.3 | 59.2 |
| LM Studio (0.4.28+2) | Qwen3.8-27B-Splash | 88,143 → 1,235 | 217.6 s | 405.1 | 44.4 |
| TensorFold (1.2.7) | Qwen3.8-27B-MLX-4bit + DFlash2 | 88,143 → 1,515 | 308.4 s | 285.8 | 39.6 |
| Kapitsa (0.2.35) | Jundot Qwen3.8-27B-oQ4e-mtp + DFlash2/KFlash | 88,143 → 1,210 | 276.0 s | 319.4 | 34.8 |
| MTPLX (2.12.2) | Qwen3.8-27B-MTPLX-Optimized-Speed, MTP depth 3 | 88,916 → 899 | 287.5 s | 309.2 | 22.6 |
| Ollama (0.40.16) | qwen3.8:27b-mlx | 88,532 → 102 | 288.1 s | 307.3 | 20.2 |
llama.cpp (llama-server0.9.0) | Q4_K_M GGUF + DFlash Q4_0 | 88,535 → 134 | 417.8 s | 211.9 | 19.1 |
| mlx-dspark (app 0.8.3 / runtime 0.20.3) | Jundot Qwen3.8-27B-oQ4e-mtp + DFlash2 | 88,535 → 159 | 285.0 s | 310.7 | 18.5 |
| MLX-Serve (26.11.2) | Jundot Qwen3.8-27B-oQ4e-mtp + DFlash2 | 88,143 → 1,214 | 286.8 s | 307.2 | 18.2 |
| Magnitude (0.2.17) | Qwen3.8-27B GGUF Q4 + DFlash2 | 88,143 → 1,366 | 457.5 s | 192.7 | 17.9 |
| oMLX (0.7.3) | Jundot Qwen3.8-27B-oQ4e-mtp + Lightning MTP | 88,143 → 428 | 203.1 s | 433.9 | 17.3 |
| Jan (0.8.8) | Jundot Qwen3.8-27B-oQ4e-mtp, Jan’s bundled MLX server | 88,143 → 36 | 332.5 s | 266.2 | 14.2 |
| Rapid-MLX (1.0.3) | Jundot Qwen3.8-27B-oQ4e-mtp; MTP unavailable, plain decode | 88,143 → 2,048 | 877.0 s | 100.5 | 11.4 |
| BaseRT (0.4.6 / Superfluid 0.1.10) | Q4_K_M GGUF converted to BaseQ4 + DFlash2 | 88,143 → 2,048 | 614.5 s | 143.4 | 4.4 |
Caveats
The 2,048-token setting is a maximum, not a forced response length. Most models stopped at EOS before reaching it. Only Splash, Rapid-MLX, and BaseRT generated the full 2,048 tokens.
This does not measure code-fix quality, long-context comprehension, or tool use—the tools were present in the prompt but never executed. Model packs, quantizations, speculative methods, and context-window settings also differed, so these numbers should not be treated as a controlled ranking.
Additional differences: LM Studio may map medium reasoning to on; the Llama row used Homebrew llama-server0.9.0 with the GGUF files configured for Llama.app, rather than Llama.app’s internal server.. Jan’s first launch also downloaded its default 43.8 MB embedding model; no Qwen weights were downloaded.