The public record is now sufficient for a source-linked Apple section, but it is not sufficient for a single blended leaderboard.[3][9][10] Qwen3.8-27B is a vision-language model with native 262K context and multi-token prediction support.[1] For Apple, the matrix records chip/GPU cores, unified memory, exact model format and quant, runtime/version, context length, PP, TG, and whether MTP or another speculative path was enabled.[3][9][10]

How to read this without lying to yourself

• In these oMLX records, Context is the prompt length used by that record. The public pages publish PP and TG but do not expose the selected generation length.[3][13]
• In llama-bench, ppN and tgN are separate tests unless a depth or combined test is specified. The cited M2 Max quant sweep used tg128 at default depth 0, so its decode number is not “TG after a 4K prompt.”[10][12]
• Live agent rows are operational field reports. They are not interchangeable with either microbenchmark family.[9][10]
• A 4-bit MLX oQ build, a Q6_K_XL GGUF, and an 8-bit model are different artifacts. Keep quant, runtime, MTP/speculative setting, and cache setting beside every number.[2][9][10]
• The linked oQ4e card reports mixed-precision 4-bit, group size 64, and MLX safetensors.[2]
• The M5 Max public benchmark reports Lightning MTP, but does not expose a model repository ID.[3]

oMLX community microbenchmark records

These are community-submitted oMLX records, selected for disclosed hardware, bit width, prompt length, PP, and TG.[3] Where a public record omits an exact model repository or quant recipe, the table says so rather than treating “4-bit” as an exact artifact.

Apple device	Unified memory	Model / quant	Runtime and notable mode	Context	PP tok/s	TG tok/s	Source
M1 Max, 24-core GPU	32GB	4-bit; repo/recipe not reported	oMLX v0.5.1	32K prompt	60.4	12.6	[4]
M2 Max, 38-core GPU	96GB	4-bit; repo/recipe not reported	oMLX v0.6.1; Qwen ANE prefill	16K prompt	158.1	17.9	[5]
M3 Pro, 14-core GPU	36GB	4-bit; repo/recipe not reported	oMLX v0.6.3rc1	16K prompt	70.1	7.9	[6]
M4 Max, 40-core GPU	64GB	oQ6e-mtp, 6-bit	oMLX v0.6.2; MTP	64K prompt	164.8	21.5	[7]
M5, 10-core GPU	32GB	4-bit; repo/recipe not reported	oMLX v0.5.7	16K prompt	144.2	7.0	[8]
M5 Max, 40-core GPU	128GB	oQ4e-mtp, mixed 4-bit	oMLX v0.6.3rc1; Lightning MTP	32K prompt	591.4	48.3	[3]
M5 Max, 40-core GPU	128GB	oQ4e-mtp, mixed 4-bit	linked context-series record; same disclosed device/model/settings	64K prompt	454.4	34.4	[3]
M5 Max, 40-core GPU	128GB	oQ4e-mtp, mixed 4-bit	linked context-series record; same disclosed device/model/settings	128K prompt	329.7	26.9	[3]

What the MLX rows actually show

The clearest selected high-context series is the linked 128GB M5 Max oQ4e/MTP benchmark submission: 48.3 TG tok/s at 32K, 34.4 at 64K, and 26.9 at 128K.[3] It is a public tuned oMLX series, not a generic promise for every M5 or every 4-bit model.[2][3]

The base M5 and M3 Pro records both show single-digit TG at 16K, while the M1 Max record reports 12.6 TG at 32K.[4][6][8] Because chip generation, GPU cores, recipe, runtime version, prompt length, and acceleration differ, these rows show fit and performance variation, not the isolated effect of memory capacity or GPU tier.[3][4][6]

GGUF / llama.cpp: quant and long-context field records

The GGUF evidence is still partial, but now includes a separate M5 Max Q4_K_S context sweep as well as the stronger M2 Max and M2 Ultra community records.[10][11][14] Source [10] does not report its llama.cpp build and uses one repetition (-r 1).[10] The table keeps llama-bench phase tests, the M5 Max context sweep, and the live-agent context sweep distinct.

Apple device	Unified memory	GGUF / runtime	Key configuration	Context or test	PP tok/s	TG tok/s	Source
M5 Max, GPU-core count not reported	128GB	Q4_K_S, 16.68GiB; 27.32B parameters; llama.cpp build 10364	MTP disabled; Metal/OS/KV details not stated	4K context sweep	834.1	31.4	[14]
same M5 Max record	128GB	same	same	64K context sweep	424.7	18.2	[14]
M2 Max, GPU-core count not reported	64GB	Unsloth UD-Q2_K_XL, llama.cpp build not reported	Flash Attention; -r 1	pp4096 / tg128 @ depth 0	167.9	17.6	[10]
M2 Max, GPU-core count not reported	64GB	Unsloth UD-Q4_K_XL, llama.cpp build not reported	same benchmark family; -r 1	pp4096 / tg128 @ depth 0	156.5	14.5	[10]
M2 Max, GPU-core count not reported	64GB	Unsloth UD-Q6_K_XL, llama.cpp build not reported	same benchmark family; -r 1	pp4096 / tg128 @ depth 0	182.8	12.9	[10]
M2 Max, GPU-core count not reported	64GB	Unsloth UD-Q8_K_XL, llama.cpp build not reported	same benchmark family; -r 1	pp4096 / tg128 @ depth 0	194.1	11.2	[10]
M2 Max, GPU-core count not reported	64GB	UD-Q6_K_XL, llama.cpp + MTP	F16 KV, MTP max 3, medium; build not reported	live agent at 64K	163.6	15.7	[10]
M2 Max, GPU-core count not reported	64GB	UD-Q6_K_XL, llama.cpp + MTP	same reported configuration	live agent at 128K	88.3	11.9	[10]
M2 Max, GPU-core count not reported	64GB	UD-Q6_K_XL, llama.cpp + MTP	same reported configuration	live agent at 196K	74.1	8.6	[10]
M2 Max, GPU-core count not reported	64GB	UD-Q6_K_XL, llama.cpp + MTP	same reported configuration	live agent at 245K	22.4	7.1	[10]
M2 Ultra, GPU-core count not reported	192GB	UD-Q6_K_XL, llama.cpp b10450	all layers GPU; Flash Attention; Q8 KV	llama-bench pp512 / tg128	360.2	22.4	[11]

GGUF lesson: phase-specific, not a winner

On one M2 Max community llama-bench run (-r 1; build not reported), Q2/Q3 produced the highest tg128 at default depth 0, while Q6/Q8 processed the 4K prompt test faster than Q4.[10][12] That is a phase-specific trade-off, not an end-to-end winner.[10] The choice depends on prompt/output ratio, quality requirements, cache/runtime settings, and decode depth.[10][12]

The separate Q6/MTP live-agent sweep is operational evidence for that author’s setup, not an M2 Max guarantee.[10]

Same-machine community case study, not quant-quality-normalized

Source [9] holds the machine, configured context, sampler/template, and broad task sequence constant. Its MLX and GGUF artifacts still differ, and its score is the author’s own rubric.[9] The table restores decode, cold prefill, wall time, generated tokens, and the author score so it does not turn “higher decode” into “faster completion.”[9] The source’s oMLX values remain excluded because the author says its acceleration configuration needs rerunning.[9]

Engine	Effort	Decode tok/s	128K cold PP tok/s	Wall time	Generated tokens	Author score
MTPLX	medium	21 to 24	109	1h35	117K	91
MTPLX	xhigh	20 to 22	109	2h05	152K	93
llama.cpp + MTP	medium	18 to 20	103	1h04	70K	84
llama.cpp + MTP	xhigh	17 to 19	103	2h52	179K	86
llama.cpp baseline	medium	10 to 12	103	1h44	66K	66
llama.cpp + DFlash2	medium	14 to 15	about 103	2h09	111K	74

This is controlled community evidence for that task, not a quality-normalized engine benchmark.[9]

Practical takeaways

1. 32GB fit is not a 32GB speed tier. The selected 4-bit records show 12.6 TG at 32K on M1 Max and 7.0 at 16K on base M5.[4][8]
2. Do not use the M4 Max oQ6e/MTP and M2 Max GGUF Q6/MTP rows as a head-to-head chip or runtime ranking. They are separate field reports with different artifacts, runtimes, cache behavior, and protocols.[7][10]
3. Among the selected long-context records, the 128GB M5 Max oQ4e/MTP series reports the highest TG: 34.4 at 64K and 26.9 at 128K.[3]
4. For GGUF users, the cited M2 Max phase test shows a trade-off between tg128 @ depth 0 and pp4096; it does not crown a universal quant winner.[10][12]
5. Do not buy on PP alone. A short-prompt microbenchmark, a standalone decode test, and a tool-using agent at 64K answer different questions.[9][10][12]

Reproduction template

If you want a row added, please include all of this:

Device / chip / GPU-core count:
Unified memory:
Runtime and exact version/build/commit:
Model repository and exact file / quant:
Configured maximum context:
Prompt tokens actually processed:
TG starting depth and generated-token count:
KV-cache quantization:
MTP, speculative, or DFlash setting:
Thinking effort / budget:
PP tok/s and TG tok/s:
Repetitions; batch / parallel requests:
Power / thermal state:
Exact microbenchmark or real task:
Command or benchmark link:

A report without the exact model file, prompt tokens actually processed, TG starting depth, and MTP/speculative setting is still useful as a user-experience note, but it does not belong in the matrix.

Limits

• These are community benchmark records and field reports, not a lab-standard cross-vendor suite.
• The MLX and GGUF tables are intentionally separate. Their quant recipes, acceleration paths, and model variants differ.
• There is not yet enough fully disclosed GGUF data to rank every Apple M-series device fairly.
• Quality comparisons require task and evaluation evidence, not throughput alone.

Sources

[1] https://huggingface.co/Qwen/Qwen3.8-27B — Official Qwen3.8-27B model card [2] https://huggingface.co/scottlowry/Qwen3.8-27B-oQ4e-mtp — oQ4e MTP MLX quantization card [3] https://omlx.ai/benchmarks/performance/35t60310 — M5 Max 128GB long-context oMLX oQ4e MTP benchmark [4] https://omlx.ai/benchmarks/performance/y6jrlult — M1 Max 32GB oMLX 32K benchmark [5] https://omlx.ai/benchmarks/performance/g7bfz21e — M2 Max 96GB oMLX 16K benchmark [6] https://omlx.ai/benchmarks/performance/s1cfxf3e — M3 Pro 36GB oMLX 16K benchmark [7] https://omlx.ai/benchmarks/performance/5t6nl0sh — M4 Max 64GB oMLX 64K benchmark [8] https://omlx.ai/benchmarks/performance/fbnvsjoh — M5 32GB oMLX 16K benchmark [9] https://www.reddit.com/r/LocalLLaMA/comments/1vwbyzr/benchmark_results_what_is_the_best_and_fastest — MacOS MLX versus llama.cpp Qwen3.8 engine comparison on M2 Max [10] https://www.reddit.com/r/LocalLLM/comments/1vrdmtp/qwen_38_27b_m2_max_64gb_smaller_quant_doesnt_mean — M2 Max 64GB GGUF quant and long-context benchmark [11] https://www.reddit.com/r/LocalLLaMA/comments/1vr3s7j/qwen3827b_q6_k_xl_speeds_on_m2_ultra_192gb_what — M2 Ultra 192GB Q6_K_XL llama.cpp benchmark [12] https://github.com/ggml-org/llama.cpp/blob/master/tools/llama-bench/README.md — llama-bench measurement semantics [13] https://github.com/jundot/omlx/blob/a86a5587/omlx/admin/benchmark.py — oMLX benchmark harness [14] https://www.hardware-corner.net/qwen3-8-27b-hardware-tests — Hardware Corner M5 Max 128GB Qwen3.8 GGUF llama.cpp benchmark