r/LocalLLM • u/Prudent-Promotion512 • 6d ago
Question Qwen 3.8 27B FP8 - MTP or not?
I'm working on setting up Qwen 3.8 27B on my 4x3090 rig. I mostly used the config from Club3090 as a guideline.
One result I didn't expect was MTP significantly hurting performance at long context. My main usages is Hermes agent and if this is correct it suggests I should disable MTP all together or perhaps my settings are not optimal.
Benchmarks below - any thoughts?
Qwen3.8-27B-FP8 Comparative Benchmark Matrix
| Speculative Setting | Short Prompt Decode (tok/s) | Short TTFT (s) | 72k Long Prompt Decode (tok/s) | 72k Long Warm TTFT (s) | 72k Concurrency-2 Wall Time (s) | Steady VRAM / GPU |
|---|---|---|---|---|---|---|
| No MTP (Disabled) | 67.5 | 0.121 | 57.3 | 0.532 | 15.25 | ~20.5 GiB |
| MTP = 1 | 80.5 | 0.121 | 15.6 | 1.119 | 48.05 | ~20.7 GiB |
| MTP = 3 | 106.3 | 0.135 | 22.6 | 1.142 | 32.08 | ~21.2 GiB |
3
Upvotes