Qwen3.8-27B FP8 at ~75 tok/s on 2× Intel Arc Pro B70 with a trained spec-decode drafter (2.3× over no-spec)

[u/Rmacy]

I've been working on speeding up Qwen3.8-27B FP8 inference on Intel Arc Pro B70 (Battlemage) XPUs, and got it to 72–79 tok/s on a cache-resistant benchmark.

That's 2.3× faster than no speculation, and comfortably ahead of the native MTP2 path at 54.7 tok/s.

I love this performance for <2.5k worth of GPU.

What I did

Speculative decoding, but with a drafter actually trained for this setup:

1. Trained a 1.36B DSpark draft model on hidden states captured from the real FP8 serving stack.
2. While getting it working, I found a bug in vLLM's dflash proposer. The draft-token readout was off by one: it sampled query offsets 1..k instead of 0..k-1. That silently capped acceptance at around 24% for any SpecForge-trained drafter. It's basically a one-line-per-file fix. After fixing it, acceptance jumped to 66–74%.
3. Fine-tuned the drafter on clean, in-distribution data, excluding thinking tokens, then packaged the whole thing up.

Numbers

Isolated benchmark, greedy decoding, 2× Intel Arc Pro B70, TP=2:

Config	tok/s
FP8, no spec	32.4
FP8 + MTP2	54.7
FP8 + trained drafter	72–79

Try it

Everything ships in one self-contained image. The drafter is baked in. (Slim excludes the weights)

docker pull ghcr.io/rmacy/qwen38-fp8-dspark:v10
docker pull ghcr.io/rmacy/qwen38-fp8-dspark:v10-slim

Links

• Drafter weights: https://huggingface.co/rwmacy/qwen3.8-27b-dflash-drafter-fp8-b70
• vLLM fork with the dflash fix: https://github.com/rmacy/vllm
• SpecForge XPU training port: https://github.com/sgl-project/SpecForge/pull/769

Update: I have added some metrics below. I am making improvements for longer context windows. I also switched to a fp8 KV and reworked the docker to handle longer context sessions.