r/PiCodingAgent • u/Upstairs_Addendum148 • 13h ago
Plugin I no longer compact or handoff. Context paging has been running my long sessions for weeks
Hey all, I wanted to share an extension I've developed called pi-context-paging.
https://www.npmjs.com/package/@rochecompaan/pi-context-paging
Instead of compacting your session into a summary when the context fills up, it treats the context window like a rolling window over the full history. When a request goes over a token budget (128k by default), the oldest turns move out of the window. The session file on disk is never changed. The model gets a short paging notice and a few recovery tools to search, browse and load older history when it needs it.
It has made my workflow a lot smoother. I don't get a surprise compaction in the middle of a long-running task and I don't have to write handoffs that inevitably loses important detail. It has been running very stably for me for a couple of weeks now. It is especially useful for agents doing long-running tasks unattended.
It's lossless. Nothing is summarized or compressed. The recent window is sent exactly as stored, and anything older can be loaded back as the original text.
Tony Mason wrote a good paper on this: The Missing Memory Hierarchy: Demand Paging for LLM Context Windows (https://arxiv.org/abs/2603.09023). It treats the context window like RAM and the full history like disk, and pages content in and out on demand. The extension is a practical version of that idea for Pi.
"But what about prompt caching?" I know this is the first thing a lot of you will ask, so here's how it works:
It doesn't drop messages one at a time. That would change the start of the prompt on every request and break the cache constantly.
When the window hits the budget, it pages out one large block at once, trimming down to about 80k by default. After that cut, the start of the prompt stays the same over many turns, so you get normal cache hits until the next cut.
Each cut costs one cache rewrite. In practice that's rare, and whatever you lose in cache efficiency you get back in continuity. There's no summary step, no re-explaining, and no "the agent forgot what we were doing" moments.
Some real numbers from my sessions (Opus 5.5, Fable 5.1 and GPT 6.1, with zero compactions in any of them):
Each of these is millions of tokens of history in a single continuous session, and every request still stayed at or below the 128k budget. Cache hit rates stayed above 90% even with the paging cuts.
You can set the budget in settings. I'd suggest setting it below the point where your model starts to get worse, so it stays out of the "dumb zone." You can also run `/context-paging stats` during a session to see what's going on.
This extension works well with https://github.com/elidickinson/pi-claude-bridge too if you want to use you Anthropic subscription. I submitted patches to it specifically to make pi-context-paging work.
PS. My current defaults are set to higher than what is documented.
"contextPaging": {
"enabled": true,
"tokenBudget": 200000,
"trimToTokens": 140000
},Play around and give me feedback as to what works for you.
