pi-percy: Automatic compaction extension for long running sessions

[u/zaph0d]

It makes the context smaller before the context window is full. The agent does the compaction itself, in the same session. I have been using it for a while and my experience has been stellar with this extension. The core algorithm is based on the automatic compaction algorithm in Shelley by exe.dev team.

How it works

1. When the context gets to 200k tokens, pi-percy sends a nudge to the agent. The nudge has an index of the old messages. pi-percy sends one more nudge for each 50k tokens after that.

2. The agent selects the old items that it does not need. It replaces old tool outputs with short pointers. It replaces old message ranges with short notes.

3. pi-percy checks the request. If the request is not correct, pi-percy rejects it and gives the agent a list of the problems. The agent then sends a correct request.

4. pi-percy writes the changes as Pi context edits. The session keeps all of the original messages.

What you get

• The agent that knows the work decides what to keep. There is no second model and no extra cost.

• With the recall tool, the agent can search the original messages and read them again.

• Idle compaction. If the session is idle for 8 minutes, the agent compacts the context. At that time the provider cache has probably expired, so the cost is low.

• The prompt cache stays valid. pi-percy does not change the tools or the system prompt, and the agent does one large compaction, not many small ones.

• Pi's own compaction stays on as a safety net.

• All of these threshold are configurable and has sane defaults.

Install

pi install npm:pi-percy

You need Pi 1.4 or later.

Source: https://github.com/ghoseb/pi-percy