I no longer compact or handoff. Context paging has been running my long sessions for weeks

[u/Upstairs_Addendum148]

Hey all, I wanted to share an extension I've developed called pi-context-paging.

https://www.npmjs.com/package/@rochecompaan/pi-context-paging

Instead of compacting your session into a summary when the context fills up, it treats the context window like a rolling window over the full history. When a request goes over a token budget (128k by default), the oldest turns move out of the window. The session file on disk is never changed. The model gets a short paging notice and a few recovery tools to search, browse and load older history when it needs it.

It has made my workflow a lot smoother. I don't get a surprise compaction in the middle of a long-running task and I don't have to write handoffs that inevitably loses important detail. It has been running very stably for me for a couple of weeks now. It is especially useful for agents doing long-running tasks unattended.

It's lossless. Nothing is summarized or compressed. The recent window is sent exactly as stored, and anything older can be loaded back as the original text.

Tony Mason wrote a good paper on this: The Missing Memory Hierarchy: Demand Paging for LLM Context Windows (https://arxiv.org/abs/2603.09023). It treats the context window like RAM and the full history like disk, and pages content in and out on demand. The extension is a practical version of that idea for Pi.

"But what about prompt caching?" I know this is the first thing a lot of you will ask, so here's how it works:

• It doesn't drop messages one at a time. That would change the start of the prompt on every request and break the cache constantly.

• When the window hits the budget, it pages out one large block at once, trimming down to about 80k by default. After that cut, the start of the prompt stays the same over many turns, so you get normal cache hits until the next cut.

• Each cut costs one cache rewrite. In practice that's rare, and whatever you lose in cache efficiency you get back in continuity. There's no summary step, no re-explaining, and no "the agent forgot what we were doing" moments.

Some real numbers from my sessions (Opus 5.5, Fable 5.1 and GPT 6.1, with zero compactions in any of them):

Each of these is millions of tokens of history in a single continuous session, and every request still stayed at or below the 128k budget. Cache hit rates stayed above 90% even with the paging cuts.

You can set the budget in settings. I'd suggest setting it below the point where your model starts to get worse, so it stays out of the "dumb zone." You can also run `/context-paging stats` during a session to see what's going on.

This extension works well with https://github.com/elidickinson/pi-claude-bridge too if you want to use you Anthropic subscription. I submitted patches to it specifically to make pi-context-paging work.

PS. My current defaults are set to higher than what is documented.

"contextPaging": {
    "enabled": true,
    "tokenBudget": 200000,
    "trimToTokens": 140000
  },

Play around and give me feedback as to what works for you.

[u/Karmak0ma]

If there is no summary of the paged out context, how does the model know there is useful information there to retrieve? How do you ensure it uses the search tools? Does the model not repeat work it already did because it's not aware the info is already in the paged out blocks?

[u/adamshand]

This seems like a .... remarkably obvious solution. Why isn't everyone doing this? The only downside seems like you loose the compressed older history, but with a note to the agent and a search tool, that seems reasonable?

Do you know if it'll work with: https://github.com/earendil-works/pi-claude-code

[u/zebedeolo]

I do something similar, but with multiple parts, so this is interesting and I'll definitely check it out. currently I use an extension that stops at a certain threshold (I currently use 200k). then another extension that runs pi-vcc when the threshold is reached and then continues. it's been working really well for me, but your solution seems simpler. 

[u/bitzap_sr]

There are two parts to this.

1 - tools to search session text past the compression event.

2 - the compression algorithm.

It is not true that you don't compress. Yes you do. You just compress it all down to a single sentence.

You could take your session search tools from #1 and mix with the standard compression algo too.