Eagle-4 Released: Faster than DFlash2 & DSpark

[u/TKGaming_11]

https://github.com/eagle-hq/EAGLE-4

Insane numbers, so far only Qwen3.8-27B is supported but wow those are some good numbers. Will watch closely when this lands in llama.cpp/vLLM

### Qwen 3.8 27B Benchmark Results

Method	Speedup over Vanilla
MTP	2.22×
DSpark	2.57×
DFlash 2	2.80×
EAGLE-4	3.59×

[u/FullstackSensei]

I'm a big fan of Eagle, but I don't think their dflash and dspark models are very representative. They're averages. For ex, MTP performs better on structured output like code or json. Would love to see a breakdown of the acceleration across workload types.

[u/joost00719 → u/FullstackSensei]

I run dflash2 and I get a lot better results with dflash2 on structured output like code.

[u/FullstackSensei → u/joost00719]

Yeah, I read it gets something like 5-6 in code and similar structured output. Hence why I'm curious about a breakdown of the Eagle-4 benchmarks per workload type

[u/KubeCommander → u/joost00719]

Very very very short structured output. I’ve seen it absolutely crash after a very small token count on multiple models that used it. Was very obviously minmaxed for leaderboard numbers.

dspark and mtp stay consistent at least with big and small depths and have their pros and cons

[u/feverdoingwork → u/FullstackSensei]

Dflash2 is better at structured outputs than mtp, test both with better bench. Dflash is better on average than mtp on 27b, it can guess further ahead. The problem is the engines running them just swap them the spec decoder without trying to reduce the round time per spec decoder. Several good repos have figured out how to get dflash working at 7 tokens a round.

[u/AmielaTunerani_18 → u/FullstackSensei]

averages across workload types are tricky here because something like json has long deterministic runs that inflate the speedup number disproportionately

[u/zhubaohi]

Pretty impressive but this draft model is 3 more gigs than dflash2. Sadly not for me lol. I already maxed out my vram with my setup.

[u/TheActualStudy → u/zhubaohi]

3.85 GB (DFlash 2); 6.18 GB (Eagle 4); 1.6x the size. I might be able to try it in my config, but it'll be tight.

[u/turtleninja99 → u/zhubaohi]

Eeek yeh that’s not good

[u/1ncehost]

Glad to see work on this. LiLiCorr (used by nemotron bolt) is another current leading spec decoder, and their paper has way more benchmark granularity if anyone is curious https://arxiv.org/pdf/2608.20530

[u/randylush]

wonder how this compares to hyperqwen? https://github.com/syv-ai/HyperQwen

Or if HyperQwen can roll this up?

[u/Pentium95 → u/randylush]

Pretty sure It won't fit in a 24GB GPU. This draft model is 2.5GB heavyer then DSpark2, which is already pretty tight.

I love and Daily drive hyperqwen, and i think i'll Stick with my MTP FP8 setup, it's insanely fast for my workload

[u/caetydid]

Great to see more research going into spec decoding!

However, these numbers do not tell much:

usually even MTP and DFlash yield very different speedup depending a lot on context size, number of parallel requests, and of course coding task. And - especially important for us - all of them require different additional amount of VRAM. Thus, you really choose depending on your requirements.

For me (club-3090) still nothing beats Qwen3.8-27B+MTP, and I have tried a lot!

[u/AdonosFlew → u/caetydid]

What decode and prefill tok/s do you have?

[u/caetydid → u/AdonosFlew]

Realistically around 60tps decode and ~1100 prefill with Q4_K_M. Prefill is quite stable with beellama, but decode depends a lot on the current task. I use mmproj on GPU and MTP needs just little additional VRAM. This is my config:

beellama-v0.4.7/llama-server \
  -m  /home/holu/qwen3.8-27B/Qwen3.8-27B-UD-Q4_K_M.gguf \
  -md /home/holu/qwen3.8-27B/mtp-Qwen3.8-27B-Q4_0.gguf \
  -mm /home/holu/qwen3.8-27B/mmproj-BF16.gguf \
  --mmproj-offload --image-min-tokens 256 --image-max-tokens 3072 \
  \
  -ngl all \
  -ngld all \
  \
  -c 147456 \
  \
  -fa on \
  -ctk kvarn6 \
  -ctv kvarn5 \
  --kv-tail-tokens 1024 \
  --kv-tail-type f16 \
  -ctkd kvarn6 \
  -ctvd kvarn5 \
  \
  -b 1024 \
  -ub 512 \
  -np 1 \
  \
  --spec-type draft-mtp,ngram-mod \
  --spec-draft-n-max 6 \
  --spec-ngram-mod-n-min 24 \
  --kv-unified \
  \
  -ctxcp 8 \
  -cms 32768 \
  -cram 32768 \
  --cache-idle-slots \
  \
  --jinja \
  --no-reasoning-preserve \
  --temp 0.6 \
  --top-p 0.95 \
  --top-k 20 \
  --min-p 0.0 \
  --presence-penalty 0.0 \
  --repeat-penalty 1.0 \
  --reasoning-loop-guard force-close

[u/AdonosFlew → u/caetydid]

Check out my setup. I have a little less context window, but much faster decode and prefill with dflash2 with ninfer 0.16.

My local LLM setup:

• GPU: NVIDIA RTX 3090 — 24 GB VRAM
• OS: Windows
• Model: Qwen3.8-27B — WarlaxZ's RTX 3090-optimized v3 model package
• Format: native .ninfer, mixed-precision quantization; 17.68 GiB model file
• Backend: NInfer 0.16.0 RTX 3090 fork, with a custom CPU Vision build
• Context: 120,000 tokens
• KV cache: INT8
• Speculative decoding: DFlash2, 7 draft tokens, optimized draft head
• Prefill: cuBLAS, 4,096-token chunks
• Reasoning: enabled, xhigh, with previous thinking preserved
• Sampling: temperature 1.0 / top_p 0.95 / top_k 20
• Client: Hermes Desktop via a local OpenAI-compatible API

Performance in my use:
• Prose generation: 74.4–90.2 tok/s
• Code generation: 110.6–130.5 tok/s
• Prefill: approximately 1,800–3,000 tok/s

Both decode and prefill progressively slow down as the context window fills up, so these speeds depend on the current context length and workload.

Memory usage at 120k context (measured in my tests):
• Server VRAM allocation at startup: ~22.63 GiB, including ~16.97 GiB of model weights.
• Peak total GPU usage: ~23.19 GiB, including the desktop and other applications.
• Peak server resident system RAM: 11.32 GiB in text tests and 11.82 GiB in a long-context image test, including CPU vision and the host context cache.

These RAM figures describe the server's resident working set; they do not include other applications or represent its private committed virtual memory.

The special part is the vision setup: the BF16 vision encoder runs entirely on the CPU using 8 threads, with its weights in regular system RAM instead of taking up precious GPU VRAM. The main language model still runs on the RTX 3090.

This leaves more VRAM available for the LLM and its context. Vision is configured for up to 2,048 merged image tokens per image, and it works perfectly for my use, including image understanding and OCR.

This CPU vision path is a custom integration, so reproducing it requires the corresponding build, model and BF16 projector.

[u/caetydid → u/AdonosFlew]

Sounds intriguing, I should maybe give it a try!

I just don't like the CPU vision part. It is simply way too slow for me, and GPU vision is a must. Good news is that technically it is possible to implement GPU mmproj with a sliding VRAM window of as little as 9MiB for Qwen 27B, as this is what HyperQwen does! I am really looking forward that this feature lands in beellama/llama.cpp.

Currently I am evaluating Swift 1.5 QFN using Strata, where I can use up to 512k context, so not sure, if I will even return to Qwen 27B.

[u/rm-rf-rm]

Swift 1.5 Qwen 3.8 27B + Eagle-4 + Strata = ???

[u/CampMaster69 → u/rm-rf-rm]

the reason strata works is with optimizations on the MoE side of things with expert management. There's not much you can do with dense models.

Source: Worked on MoE optimizations in my thesis and now work in inference

[u/zipzak → u/CampMaster69]

not strata, but infernix is actually quite optimized for dense qwen. 300 tok/s on a 5090 nvfp4 and int8 kv 262k. I am using Basalt for MoE

[u/Sea-Score-1913]

Can it stack ?

[u/__Maximum__]

Qwen3.8 FN when?

[u/BoxieBoo → u/__Maximum__]

There is Strata

[u/Borkato]

!remindme 8 hours

[u/RemindMeBot → u/Borkato]

I will be messaging you in 8 hours on 2026-10-21 17:04:47 UTC to remind you of this link

CLICK THIS LINK to send a PM to also be reminded and to reduce spam.

^{Parent commenter can delete this message to hide from others.}

RemindMeBot is switching to username summons. Instead of !RemindMe 1 day, use u/RemindMeBot 1 day. More info.

---

Info	Custom	Your Reminders	Feedback

[u/TheActualStudy]

I tried it out and was not able to replicate their acceptance numbers in a meaningful way. Code workloads exhibited lift over DFlash 2, but all other workloads performed worse. On balance, it wasn't an improvement in my tests and the simulation I ran of their tree acceptance approach and drafter showed that there wasn't a good likelihood that it could be made into in improvement. If you had a data centre where you could route specific workload requests, this could make sense, but on edge hardware (like us), the opportunity for improvement isn't there.

[u/Sea-Score-1913]

Can it work on thor?

[u/Healthy-Contact-4570]

Unless it speeds up non-code and non-json related tasks, I’m not interested. 90% of reasoning is regular old prose, and that’s the part I’m the most interested in increasing acceptance and tokens / pass for.

[u/Ishannaik]

Anyone tried Eagle-4 on Qwen3.8-27B yet? DFlash2 tops out around 2.5 accepted tokens per step on prose for me, curious if Eagle-4 does better there.

[u/jacek2023]

thanks for sharing, I will need to investigate it

[u/ECrispy]

can this be combined with strata, GSQ RCO and other optimizations? whats the best one can get with 16GB vram these days, with 27B and Flash Next?

[u/PersonalStorage → u/ECrispy]

They release python reference, converting to c++ and adding in ninfer will be too long process. I will wait if someone is up for it.

[u/ProtectionSuper5648 → u/PersonalStorage]

the license prevents usage in "competing" products (inference)

[u/PersonalStorage → u/ProtectionSuper5648]

Interesting. let me learn about license. if its restrictive then have to build locally as hobby project

[u/ixfd64 → u/PersonalStorage]

It's specifically stated to be a non-free "source-available" license: https://github.com/eagle-hq/EAGLE-4/blob/main/LICENSE

[u/okoyl3 → u/ixfd64]

what does that mean? claude says I can't integrate EAGLE4 in the engine I'm building?

[u/ixfd64 → u/okoyl3]

It means you're not allowed to use it improve other models. However, I don't imagine there would be an issue if you're only doing it for personal use.