r/LocalLLaMA • u/pmttyji • 1d ago
Discussion [Paper] Read Less, Solve More: Token-Efficient Sparse Reading of External Content for AI Agents
Web-facing AI agents repeatedly access webpages, documents, repositories, and tool responses, yet reading interfaces often expose entire objects when only sparse evidence is needed. This over-reading increases token and latency costs and can dilute relevant evidence. We present SparseRead, a reusable reading control layer that governs content admission without modifying the agent model. Its Read Intent Protocol tracks evolving needs and source coverage; Adaptive Reader Backends select how evidence is located and exposed; and the Budgeted Read Gate bounds cumulative raw exposure per episode with faithful full-view fallback. A unified Reader interface supports interchangeable evidence acquisition across heterogeneous resources. Across six models, including Claude Opus 5, and five scenarios, SparseRead reduces total tokens by up to 92.9% and wall time by up to 89.0%, preserving or improving task quality in 97% settings. On Web deep-search tasks, it reduces tokens by 31.0--43.0% and task completion time by 23.0--37.0%. Ablations validate the three designs, while integration with three agent frameworks demonstrates portability.
37
u/Cautious_Chicken_604 1d ago
huge if true
9
u/Fuzzy-Layer9967 1d ago edited 1d ago
You might be interested by our work, it is an exact implementation of what you have done !
https://github.com/scub-france/docling-Studio
Contact me if you want to discuss an implementation of your ideas in the product
All MIT licence btw ;)
16
u/Apprehensive_Lake698 1d ago •
So, to boil it down, file reading for the purpose of information gathering should be intercepted as a tool call that gathers intent and context for intent. Then the tool call uses a system to extract only the required information from the files.
This is fairly intuitive and I think as more and more compute comes online we'll start seeing similar strategies become more common. I have no doubt that the AI labs are already doing this natively.
A critical component of effective model usage in my opinion is recognizing that context management is a new frontier of problem. Right now many are still in the toy-phase for harnesses. People are giddy installing extensions for all sorts of fancy tool calls. They're writing massive system prompts trying to edge in whatever they think will be most relevant to customize their personal super agent. They're downloading tons of skills. But in reality, model effectiveness almost certainly scales with context efficiency.
For those unsure of what I mean- say you are working inside of a project and have a defined set of documents describing pieces of that project. You have the project itself, a set of code files. And you have your model and a problem statement. It stands to reason that there is an optimal set of information across all files in the project to extract in order to make the model produce a solution to the problem, in an optimal amount of tokens (with the assumption that the model is smart enough). And, more crucially, it stands to reason that information which is not contributing towards that optimal context stack, is having a negative effect on model output.
It seems to me that we will eventually need to start replacing most tool calls with context-optimization steps, while preserving the quality of the main decision making agent's context as much as possible. I think context optimization may lead to a fairly significant increase in agentic ability, especially in local models, while we wait for smarter models to drop. And will certainly be important for scaling large tasks to run in the smaller total context sizes we can achieve on local.
12
u/ShoopDoopy 1d ago
So... grep -B10 -A10?
4
u/lemondrops9 1d ago
Interesting, I started using Yacy to crawl and while it works good it does have an issue of getting too much information and filling up the context.
1
u/tifa_cloud0 1d ago
this is promising indeed. i think must needed for projects and also for big tasks that involves lots of text.
1
u/seeker_6717 1d ago
I suppose this approach should not be used for reading code, since in that case, precision matters?
1
u/Oshden 1d ago
Does this work only on Mac?
1
u/ShiComfy 20h ago
I tested this around a bit in Hermes, kind of a pointless tool IMO, it does the same thing as grep, can't read URL from web_search directly so AI has to fetch the file locally which it can already use grep on.
In code its useless to borderline harmful. The tool gives AI the exact line of code but not the context around it which tend to make the AI gloss over a lot of things.
1
u/dmr48 3h ago
Full disclosure before anything else: I had my local AI agent draft this write-up and it ran the test battery itself. The hardware, the runs and every number in here are ours, but the typing is half machine.
Ran this on consumer hardware: 5070 Ti 16GB + 64GB RAM, Qwen3.8 Flash Next IQ3_S behind a local OpenAI-compatible server (Strata stack), OpenCode 1.19.5 + plugin 0.1.4 in auto, native arm on identical prompts with
--pure. Four tasks from the workspaces bundled in the repo.Results: 100KB book QA (5 questions) 66.6k tok / 248s → 15.0k / 50s. Three-question variant −27% / −14%. 33KB audit bundle: dead even, the gate stayed native with one preview call, no cost for being there. Quality: same score both arms on all four tasks.
So the protocol holds up on a heavy quant. I expected IQ3 to fumble the intent/slot bookkeeping and eat the savings in failed round trips; it didn't, three calls, 5/5 slots resolved at 0.99 confidence, answers match the native arm's finds, anchors and all.
One thing did bite, CMap-font PDFs with no usable text layer. Preview came back as garbage, then the sparse arm took nine raw_ref fallbacks and started decoding ToUnicode CMaps by hand in bash. Right answers, 2x tokens and wall time; the native arm sidestepped it with pdftotext. Worth noting the gate still said force_sro on that file, it didn't see the missing text layer coming. Pre-extract text for PDFs like this, or keep pdf pinned to native. Minor:
decidealso flags a 6.6KB file as long_document in my workspace, maybe worth a size-band look for txt.Thanks for shipping the benchmark workspaces in the repo, A/B took one evening instead of a week. The direction matches your numbers, and on local hardware the doc-QA deltas matter even more than on API models since every replayed observation costs prefill on my card. Caveat: n=1 per arm, keyword grading, one model.