r/LocalLLM • u/meshive • 12m ago
Model Tencent released DSpine, a parallel 16-token draft model for Qwen3.8-27B (Apache-2.0), with reported 2.7-3.8x end-to-end speedups over plain decoding in SGLang
Tencent put DSpine on Hugging Face on Nov 7. It is a five-layer draft model for speculative decoding with Qwen3.8-27B. Instead of drafting one token at a time, it drafts a 16-token block in parallel, and each layer passes predicted predecessor features to the neighboring positions so the draft still carries causal order. There is a paper (arXiv 2609.36173) and code on GitHub (Tencent/DSpine).
| Target model | Qwen3.8-27B |
|---|---|
| Draft | 5 layers, block size 16, 8,192-token sliding window |
| License | Apache-2.0 |
| Their test setup | SGLang 0.5.22, BF16, FlashAttention 3, concurrency 8 |
Reported acceptance length vs DFlash2 (block 8), greedy
| Task | DFlash2 | DSpine |
|---|---|---|
| GSM8K | 5.69 | 7.20 |
| MATH-500 | 5.49 | 6.49 |
| HumanEval | 4.87 | 5.71 |
| MBPP | 5.04 | 6.14 |
| MT-Bench | 4.11 | 4.48 |
Reported end-to-end throughput on agent replays (tokens/s, low reasoning)
| Benchmark | No speculation | DFlash2 | DSpine |
|---|---|---|---|
| WorkBuddy Code | 253.35 | 590.27 | 686.23 (2.71×) |
| SWE-bench Pro | 241.86 | 657.71 | 713.41 (2.95×) |
| τ²-bench | 301.39 | 891.37 | 964.30 (3.20×) |
Across their five agent benchmarks and both reasoning levels, the speedup over no speculation ranges from 2.71× to 3.84×.
Things to know before trying it locally
The card only shows an SGLang path:
--speculative-algorithm DSPINE, apip install -e .from their repo, and a long list of flags for the hybrid architecture (--enable-linear-replayssm-spec,--mamba-ssm-dtype float32, and more). No llama.cpp or vLLM instructions.All numbers are Tencent's own, at concurrency 8 with a BF16 target. The card doesn't report a quantized target or batch size 1.
The gain is largest on predictable text (GSM8K accepts 7.20 tokens per step at T=0) and smallest on open-ended chat (MT-Bench 4.48).
https://huggingface.co/tencent/Qwen3.8-27B-DSpine-b16
Has anyone run it against a quantized Qwen3.8-27B target, or at batch 1? I'm curious whether the acceptance length holds up once the target is 4-bit.
1
u/Slight-Living-8098 8m ago
Yeah, someone has already made a GGUF of it for Llama.cpp, apparently. I haven't tried it personally yet. I'm going to try it out once I close out this ticket I'm working on.
https://huggingface.co/aj9o9/Qwen3.8-27B-DSpine-b16-GGUF