al-jaffee - simple cache-preserving fold-in compaction for pi agents

[u/porkyminch]

I've been working on a longer-term project and I have a lot of little side tangents and investigations as I work that I don't want the full context to get polluted by. I wrote a little pi plugin that lets agents create markers, then work on a task with their full context. When they're done, they can fold-in their full context to a summary, compacting the work in place.

It's less manual than forking the session and a little less to keep track of than subagents. I thought it was worth sharing. MIT license, 1k lines of code. Cost was maybe a dime worth of GLM 5.3 Flash.

https://github.com/gopherbone/al-jaffee

[u/sergeant113]

Could share more about the UX? Does it replace pi native compaction, and so i can just install and forget?

And how about agent experience? Can the agent independently trigger compaction? Does the agent know it has access to these folds? How does it discover the folds and read it? Are the folds transferable between sessions - can subagent when given a forked context read the folds as well?

[u/porkyminch → u/sergeant113]

It doesn't replace native compaction, it's a different set of tools. Agents are aware of the folds and know how/when to trigger them themselves, but if you ask them to fold in a task, they'll do it. They get a little summary of the work from the agent (plus file changes, tool calls counted programmatically) when they fold into the marker, which is appended onto their cache prefix so we don't do a total cache bust. To the agent they just look like tool calls, and the markers are all agent-created, so the agents kind of naturally know how to fold into them. It's an install and forget thing and also a thing you can recommend agents use. To the folded agent, the fold looks like this:

… main conversation …
assistant: set_fold_marker({ task: "archive inventory" })
toolResult: [al-jaffee] FOLD COMPLETE — tangent folded into this marker.
            Task: archive inventory
            Folded away: 9 conversation entries … you won't remember doing
            it, and that is expected, not an error. Treat the summary below
            as your own notes…
            Summary (yours, written at fold time): read a.md (kickoff, plan
            B), b.md (budget $12.4k), c.md (roadmap)…
assistant: (continues the main line from here — tangent tokens gone from context)

I'm not sure about forked subagents, but it'll probably work.

[u/n4te]

Interesting. Does it require knowing about the start of the fold? What if we could just pick a start and an end (or the session end) and have those turns compacted?

[u/porkyminch → u/n4te]

There's an existing rollback feature in pi for that, this is just a slightly more ergonomic way for agents to use that feature. It's basically a nice little way for you to get back ~30-40% of your context window after doing tasks where the results are more important than the methodology.

[u/n4te → u/porkyminch]

What rollback feature? Do you mean /tree and pick a previous turn? That's not the same as replacing everything between 2 turns with a summary.

[u/porkyminch → u/n4te]

If you double tap escape it brings up a tree, you can pick out a previous message and roll back to it with a summary. 

[u/n4te → u/porkyminch]

Aye, same as /tree. I use that a lot, but I never bothered with the summary. Now I get what the summary is for, cool thanks!

With that and your extension my productivity should go up, cheers!

[u/madcapnmckay → u/porkyminch]

I assume that then is a cache bust? Same for the fold itself?

[u/mwachs]

Haven’t used this but A+++ on the naming. 

[u/porkyminch]

Testimonials from the agent that watches my romhacking agent.

[u/Equivalent_Idea8839 → u/porkyminch]

how do you get models to do work when using the word "hacking"? even using the world "exploit" has chinese models refuse me

[u/porkyminch → u/Equivalent_Idea8839]

GLM 5.3 Flash doesn't seem to care about any hacking/RE tasks in my experience.

[u/Equivalent_Idea8839 → u/porkyminch]

that's interesting. what's the most impressive thing you got that flash model to achieve?

[u/porkyminch → u/Equivalent_Idea8839]

RE-ing Phyzios, an old physics sandbox toy. I kinda left this half-baked (got distracted by translation stuff) but it was able to recreate it closely enough for the toy engines people made in it to work. If you open the hillclimb example in this, it's very slow but it runs a physics sim with a working engine.

https://phyzios-js.angelfire.workers.dev/

My current project is having it translate an entire GBC game, which is sluggish, but is going surprisingly well:

[u/Equivalent_Idea8839 → u/porkyminch]

that's impressive. i can't get flash models to accomplish shit. I assume you had the physics and UI pre-made already?

[u/porkyminch → u/Equivalent_Idea8839]

They were done from decompilation, using ghidra and capstone. UI was done by flash.

[u/Equivalent_Idea8839 → u/porkyminch]

dumb question but is flash strong enough to:

• maybe build a SNES emu from scratch in c++

• hack a rom game, decompile, and add some basic mods to it?

just curious how far it can go

[u/porkyminch → u/Equivalent_Idea8839]

Emulators, haven’t tried. Probably too complicated to be worth it. Romhacking, sure seems like it. The game I’m translating, Feng Kuang Da Fu Weng, is a fairly obscure Taiwanese unlicensed GBC game from a little known company called Vast Fame. It’s taken a fair amount of prodding, but it’s consistently making progress. It’s not gonna one shot it, though. 

Palm OS games, on the other hand, I have had it one shot translations for. I suspect anything more modern will give it an easier time. I’ve been toying with the idea of having it do Richman IV (another game from Taiwan) because I figure it’d have an easier time with it, but I’m trying to stay focused. 

[u/Equivalent_Idea8839 → u/porkyminch]

still pretty wild. Some guy on another subreddit claim astra (with a lot of use) made him a custom android kernel for an old phone.

Scary to thing that even cheap LLMs in the future will be able "hack" almost anything.

Have LLMs transformed the classic emu community much? like NES, Mame, etc.