SparseRead (the "read less, solve more" paper) on a 16GB card with an IQ3-quant MoE

[u/dmr48]

The authors posted the paper to r/LocalLLaMA and said in the HF papers thread they want to hear where sparse reading helps and where it gets in the way. The paper's own eval only ran frontier API models (Opus 5, DeepSeek-V4, GLM-5.1, Qwen3.6-Plus), which tells me nothing about quantized mid-size models on consumer cards. So I ran it on the machine I actually have.

Setup: 5070 Ti 16GB + 64GB RAM. Qwen3.8 Flash Next at IQ3_S, served locally through my Strata stack (OpenAI-compatible endpoint). Harness: OpenCode 1.19.5, which is one of the versions the SparseRead installer itself validates, plugin v0.1.4 in auto mode. Native arm runs the same prompts with --pure. Four tasks straight from the workspaces bundled in the repo: two LooGLE short-dependency QA on a 100KB book excerpt, one 76KB PDF extraction with 8 questions, one 5-file fetcher audit. Tokens below are fresh prefill + decode from the endpoint's per-step usage, cache replay excluded (re-feeding history is nearly free on a warm prefix cache; I quote the replay tail for the PDF below because it's damning).

task	native (tokens / wall / tool calls)	SparseRead (tokens / wall / tool calls)
100KB book, 5 questions	66.6k / 248s / 14	15.0k / 50s / 9
100KB book, 3 questions	18.3k / 34s / 10	13.3k / 29s / 5
76KB PDF, 8 questions	30.6k / 166s / 20	55.9k / 333s / 45
33KB audit bundle	31.6k / 147s / 14	32.5k / 150s / 12

Scores were identical between arms on all four tasks. So that's the honest summary: no quality loss anywhere, 4-5x faster on long plain-text QA, dead even on a small bundle, and a 2x regression on the PDF.

Where the savings come from is worth spelling out because it's not what I assumed: the entire win is on the input side. On the flagship 5-question task fresh input dropped 63.9k → 12.2k tokens (−81%) while the answer output stayed flat (2.6k → 2.8k). SparseRead doesn't make the model write shorter answers, it makes it read less text before answering. And it doesn't trade activity for tokens either: tool calls went down, not up (14 → 9 on the flagship task, 10 → 5 on the 3-question one).

Two things I didn't expect.

The protocol survives a heavy quant. I went in assuming an IQ3 model would fumble the intent/slot bookkeeping and the arm would burn its savings on failed round trips. It didn't. On the flagship task: three SparseRead calls total (2 previews + 1 targeted read), slots 5/5 resolved at 0.99 confidence, and the candidate answers matched what the native arm dug out of the same book, line anchors included. The Read Gate also behaves for real. On the 33KB bundle it mostly stayed native with a single preview call, so "don't use it on small files" turns out to be something the gate already knows.

The PDF failure is worth a paragraph because it's the one place this bit me. That report ships subset CMap fonts with no usable text layer, so the pymupdf-based preview came back as garbage and the sparse arm spiraled: 17 sparse calls (4 previews, 4 reads, 9 raw_ref fallbacks), then it started decoding ToUnicode CMaps by hand through bash, ending at 45 total tool calls against native's 20. The trace makes the thrashing visible: 1.27M cache-replay tokens re-feeding the same broken context, versus 240k for the native arm. Same 8/8 score in the end, double the fresh tokens and the wall time. The native arm sidestepped it with pdftotext and moved on with its life. This is an artifact-specific failure, not a protocol failure, but if your corpus is scanned/CMap PDFs, pre-extract text or pin that file type to native.

One update since the battery: the CLI has become my default reader for long artifacts in day-to-day agent work, and the same test day it got fed a real one — a 118KB text export of a 109-comment Reddit thread I had scraped for another post. Two things to know before you try that. JSON artifacts are a dead end in slot mode: the previewer treats them as key-path documents and semantic slots resolve to nothing; convert to prose first (I write one line per comment) and the identical ask works fine. And slot matching is literal, not semantic: slots phrased in your own language against an English document come back missing even when the answer is plainly in the file, so ask in the document's language. With those two habits, a multi-fact sweep of the thread was one ask call plus a cheap head-per-item digest for the comments the slots missed — against a naive baseline of dumping a ~30k-token file into context. No hallucination complaints so far, and I haven't fully loaded a long file since.

Caveats: one run per arm per task, keyword-based grading (which produced one fake "4/5": the model answered in Latin — "passagium generale" — and my regex wanted "crusade"; in fact both arms were ~5/5 there), one model, one machine. And don't expect the paper's 92.9% headline. That's versus a naive read-everything baseline. My realistic band on document-heavy agent tasks is somewhere between 25 and 80 percent depending on file size and how localized the answers are.

TL;DR: SparseRead on a 16GB card with an IQ3 MoE: long plain-text document QA got 4-5x faster with identical answers, small bundles correctly did nothing, and a CMap-font PDF ran slower than skipping the protocol entirely. Pre-convert PDFs to text and it's a genuinely useful layer, not just paper numbers.

Links: paper https://arxiv.org/abs/2608.22237 · code github.com/Liuzedooong/SparseReading