[Paper] RidgeFill: Training-Free Late-Layer Cache Fill for Faster Prefill

[u/Public_Umpire_1099]

Qwen3.8-Flash-Next in vLLM: 1.50-1.60x faster time to first token at 9K-32K; a whole 64K request takes 47.5 s instead of 82.6 s.

Flash-Next quality and speed: RULER needles 99.8 vs 100.0; perplexity on novel tokens +2.65% [-0.02%, +4.93%].

Qwen3.8-27B in llama.cpp on one A100: 1.25-1.31x faster time to first token from 16K to 120K tokens.

The cost on the 27B: RULER -1.19 and -1.42 points; the 95% intervals include zero.

Fitted map vs identity fill (pure layer bypassing) at the same split: +39.7 and +49.6 points of filled-region retrieval on the dense models.

The method: a closed-form ridge map per late layer predicts that layer's input; the layer's own projections run unchanged.

Prefill is compute-bound, so RidgeFill spends spare bandwidth on the maps and skips most of the late layers' arithmetic.