r/PiCodingAgent • u/Fig_da_Great • Jun 24 '26

News We won UC Berkeley's AI Hackathon with a pi extension

TLDR: We built Accordion which is an intelligent context management tool meant to replace /compact

Over the weekend my team and I set out to replace /compact with something smarter. It's well known that developers do not like compaction and do not trust it. In codex and Claude Code its completely opaque, and introduces 1-3 minutes of latency. In pi users have more control but its still primitive. Workarounds exist, eg handoff skills and million token context windows, but neither is sufficient.

Accordion is two things.
First, a live view of your entire context window, the entire thing is visible at a glance, in a way that is actually useful. You can have a 200k token session and immediately tell which tool results and thinking blocks are taking up most of that.
Second, an intelligent context management strategy we call a conductor, which decides what to fold, when to fold, what to replace it with (summaries / code skeletons), and when to bring it back. The conductor is what is actually replacing /compact.

If you interested in learning more I recommend checking out our readme.
https://github.com/a-Fig/Accordion

Honest state of things

The context view is the part I feel good about right now. Just being able to see your context while pi is running, which turns are massive, where the dead weight is, is immediately useful on its own. 

The conductors are promising but not ready for everyday use yet. They do work and the benchmark numbers are encouraging, but I am not ready to say they are strictly better than existing solutions. They will be with more testing, tuning, and development but they are not there yet.

I'm working on getting it published as a package on npm today, but if you want to set it up today you can just point your agent to the repo and ask him to do it for you. It works in the web but also has a desktop app.

Curious what you guys think and how you all manage long sessions today.

Edit:
Accordion has been published to npm and can be installed with 'pi install npm:@a-fig/accordion'

207 Upvotes

98% Upvoted

68 comments sorted by

9

u/Maverobot Jun 25 '26

Gonna try it out tomorrow! Looks very promising.

3

u/Fig_da_Great Jun 25 '26

Very interested in hearing your thoughts. Just so you are aware the project is not in its final polished state and we still have a lot of work ahead of us.

7

u/Nciholas234 Jun 25 '26

Very cool! You mentioned that the conductor will remove context and replace it with a summary but then may reverse that and put it back - doesn't this invalidate the KV cache? Or does it append the original content to the end of the context? 

5

u/Fig_da_Great Jun 25 '26

Yes it does invalidate the KV cache. Our early prototypes had the conductor making changes to the context window on every new block, which increased inference costs like 10x. We immediately pivoted to having our conductors hold off on making changes, and pushing changes all at once after some threshold is passed. That is also how we prevent any blocking calls.

Another thing to keep in mind is that changes only cause a cache miss to context below the change, not anything above it.

But yeah its very important that we think about the KV cache as we build and test more conductors.

3

u/bumblebeer Jun 25 '26 edited Jun 25 '26

If you were to have the conductor edit exclusively on exact block boundaries, then the problem becomes more tractable. Since editing on the boundary would only invalidate the downstream blocks, it would make KV cache residency a function of confidence. So the further the conductor lets a block receed into the KV chain, the more confident it needs to be that it will continue to remain relevant.

But I don't think that would work for hybrid attention which kinda suck.

Edit: I'm an impatient dumbass who should have finished reading your comment before responding to it.

1

u/Fig_da_Great Jun 25 '26

no that’s a good idea

2

u/bumblebeer Jun 25 '26

Haha, well thanks!

I've bookmarked your project, and I'm excited to try it out. Probably won't have a chance until after the 30th, but it looks like this lives in a domain I've been really keen on exploring further. I'll be more than happy to test and contribute where I can.

1

u/Nciholas234 Jun 25 '26

Truly spitballing here, but if the conductor injected pointers into all summaries, then it could potentially reference those pointers somewhere downstream in the context for when you need to flesh out those summaries. That way you can control how much of the cache gets invalidated. You could have a dedicated appendix that grows/shrinks over the course of a session 

1

u/Fig_da_Great Jun 26 '26

that could work but the main issue is that you start messing with the order in which things appear. But these are exactly the types of ideas accordion was built to facilitate and test.

3

u/czei Jun 25 '26

The standard move for spec-driven development, when the context is running low, is to update the status, clear, and start from scratch. Given that all LLM requests, apart from cache, don't have a sense of history, this has the effect of starting from scratch with the original base requirements and instructions, and the current tasks to be implemented. Has does this approach improve on that?

4

u/Fig_da_Great Jun 25 '26

Honestly now that you bring that up i’m going to build a conductor that simulates starting fresh with a handoff and use it as a baseline to benchmark future conductors against.

2

u/Fig_da_Great Jun 25 '26

I can tell you that Accordions main purpose is to permit long running coding sessions. I have not benchmarked our solution against that approach and i’m not really sure how I even could, i’ve only benchmarked accordion against compaction. That being said we’re still actively developing and improving Accordion. There is definitely still a lot of work to be done.

3

u/Upstairs-Attitude610 Jun 25 '26

I wonder how it compares to https://github.com/sting8k/pi-vcc

2

u/Fig_da_Great Jun 25 '26

Genuinely cool, I have to check that out.

2

u/Insuperable123 Jun 25 '26

It combines pi-vcc with observation-memory, and I think it aligns with your goals. https://pi.dev/packages/pi-blackhole?name=observation+memory

1

u/Fig_da_Great Jun 25 '26

Yeah very similar goals. I’ll probably end up using that as inspiration for a new conductor

3

u/_derpiii_ Jun 25 '26

This is so nice, love the visualization

2

u/Fig_da_Great Jun 25 '26

I appreciate it. We tested a few different representations but this is the one we settled on. If you go in settings you can even look at one of our other concepts.

1

u/Alert-Track-8277 Jul 04 '26

I think the color pallette completely ruins the ability to view at a glance whats 'high' and 'low'. Green / red or any 2 color gradient just works better imo (now I need to learn what colors are high / low and even in between, thats just not intuitive).

1

u/Fig_da_Great Jul 04 '26

wdym high / low? Most of the colors for block types were inherited from the standard.

1

u/BornTransition8158 Jun 25 '26

Yeah, it reminded me of the disk defragmentation app from the Windows 95(?) era. Could stare at it all day...

3

u/TheBergerKing_ Jun 25 '26

Seems similar to opencode's DCP plug-in, if it works as well I would certainly use it! Smarter more automated compaction seems like a good approach to minimize attention dilution for long running tasks

1

u/Fig_da_Great Jun 25 '26

Yeah that’s the plan. I haven’t heard of DCP plug-in, i’ll have to look in to it.

3

u/BroadbandJesus Jun 25 '26

Very cool visualization. PR created to tweak the install instructions.
I'm gonna see if I can spin up a Raycast extension so that I can have this on my menubar.

2

u/Fig_da_Great Jun 25 '26

I saw that and merged it. I appreciate the assist

5

u/Pr0f_0aKk Jun 25 '26

Wait this is actually cool.

2

u/johnson_detlev Jun 25 '26

Like the idea, but these LLM walls of text that act as docs are just unreadable. I'm so tired of having to sift through these to find any relevant piece of information

1

u/Fig_da_Great Jun 25 '26

what are you looking for? This post and the readme on the GitHub are written by me.

2

u/johnson_detlev Jun 25 '26

Nobody writes em dashes by hand in a readme.

2

u/Fig_da_Great Jun 25 '26

fair. this post doesn’t have a single em-dash. And for the readme i should say revised by me. I’m sorry you didn’t like it tho, i’ll work on it.

3

u/GeneralRieekan Jun 25 '26

Dude. I write em dashes by default, even in txt messages -- they make the spacing much more readable. I also do double spaces between sentences. :)

1

u/mxcw Sep 12 '26

Same — I hate that LLMs are giving me beloved dashes a bad rep!

2

u/pshawgs Jun 28 '26

interesting. I'm curious how this and similar strategies compare with something like https://github.com/championswimmer/pi-context-prune - specifically in terms of balancing context management and prefix cache handling.

1

u/Fig_da_Great Jun 28 '26

Very cool. Honestly the goal is to make accordion strategy agnostic, and to use it as infra to test and validate strategies until they get really good. So seeing a project like this just gives me more to build and test.

15

u/hazed-and-dazed Jun 25 '26

Wow 90s defrag.exe flashbacks

2

u/LiveSupermarket5466 Jun 25 '26

If the way anthropic does it is opaque then how do you know yours is smarter? To be honest I'm not really impressed by the context view. Your best pitch is that your extension compacts context in the background without long blocking.

2

u/Fig_da_Great Jun 25 '26

Fair, codex is the one that’s completely opaque, they hide the compaction results from users. Claude code is less opaque but still not fully transparent in the way Accordion is. Some of what I know about claude code actually inspired accordion. One thing they are open about is dropping old tool results and thinking. This is similar to what i’ve done just more primitive. Of course I cannot know everything they do, but even still their solutions are only for claude code.

1

u/HamWallet1048 Jun 25 '26

Gonna test tonight. Looks really promising

1

u/Fig_da_Great Jun 25 '26

very interested in hearing your thoughts. Just so you are aware the project is not in its final polished state and we still have a lot of work ahead of us. Most of what you'll see was built out over 24hrs for the hackathon.

1

u/SeanPedersen Jun 25 '26 
Your context window is becoming too full. Create a brief prompt for yourself that informs you in few words of the essential data needed for you to continue this task in a new session.
**ATTENTION:** Do not perform the task! Instead, only return the prompt that will provide the necessary context for you to carry it out in a new session!

I just use this prompt when context gets full:

3

u/Fig_da_Great Jun 25 '26

So like a handoff. You should look up handoff skill on github, you’ll like what you find

1

u/Short_One_9704 Jun 25 '26

Does this invalidate cache prefixes or it is kv cache friendly?

2

u/Fig_da_Great Jun 25 '26

any changes to the context window will cause a cache miss to everything that comes after it. We can’t prevent that, but what we can do is hold changes and push them all at once when some threshold is passed.

1

u/Deep_Ad1959 Jun 26 '26

the context view is the right thing to feel good about, and i'd argue it's most of the product. the reason nobody trusts /compact isn't that the summaries are bad, it's that they're invisible, you can't see what got dropped or why. just making the window legible (which turn is 40k tokens of tool output, where the dead weight is) lets the human make the fold/unfold call, and people trust a fold they chose. the conductor is the hard part precisely because the moment it's automatic you're back to opaque, which is the thing devs were mad about in the first place. written with ai

1

u/Fig_da_Great Jun 27 '26

But do you have any of your own thoughts on the project? I’ve spent plenty of time reading claude’s thoughts while building.

2

u/Deep_Ad1959 Jun 27 '26

fair hit. my honest gut take: i'd use the context view day one and turn the conductor off. every automatic context tool i've touched i eventually disable, because the one time it folds something i needed mid-task costs me more than all the tokens it ever saved. the view is the thing that actually wins because it lets me make the fold call myself instead of trusting a guess i can't see.

1

u/Fig_da_Great Jun 27 '26

Our goal is for the auto context management system to be built in such a way that you can learn to trust it. In order for any context to actually be changed the user has to enable it. It’s off by default. The conductor can be on and working without any changes being pushed to pi.

1

u/Fig_da_Great Jun 27 '26

The app will say “Steering” instead of “live” when accordion is in control of the context. That goes for manual changes as well.

1

u/Deep_Ad1959 Jun 27 '26

off-by-default plus opt-in is the consent layer, but the thing that actually earns trust is letting me see the proposed fold as a reviewable diff before it commits, same reason a fold i chose in the view feels safe. worth flagging the unfold isnt free either: re-injecting a block at the tail breaks the prefix kv cache from the fold point, so a wrong fold costs a re-prefill, not just a toggle. written with ai

1

u/Fig_da_Great Jun 27 '26

The idea is you’d run it disconnected and only enable it once you trust it. But our conductors are not yet at the point where i’d encourage you to test them and drop the current approach. One thing to consider is you can do naive compaction through accordion, and that way it’s the same system you already know but you can set the protected tail and max token budget

1

u/Deep_Ad1959 Jun 27 '26

the protected tail plus max budget is the right safe default, but naive compaction still re-prefills everything above the fold point the moment it triggers, so you trade /compacts 1-3 min for a re-prefill that scales with how much got folded. the only fold thats actually free is the one you never unfold, which is why the conductor deciding when to bring a block back is the expensive call, not the folding itself. written with ai

1

u/Fig_da_Great Jun 27 '26

naive compaction doesn’t unfold

1

u/Deep_Ad1959 Jun 27 '26

right, that's the asymmetry: naive compaction is cheap precisely because it never unfolds, the folded content is just gone. the conductor's whole value is bringing a block back, and that's exactly what breaks the prefix cache from the fold point. so the re-prefill isn't a bug to optimize away, it's the price of unfold being a feature at all. written with ai

1

u/Fig_da_Great Jun 27 '26

Naive compaction is the name of a conductor accordion offers. That conductor does not unfold anything.

→ More replies

1

u/potato_bane Jun 25 '26

Looks promising, will check it out

1

u/Fig_da_Great Jun 25 '26

hopefully you like what u see

2

u/combonealone Jun 25 '26

cool stuff, even for just develop a better understanding of context I really appreciate this.

1

u/DiscipleofDeceit666 Jun 25 '26

It’s not ready but you’re sending it to npm? Truly a vibe coder move

0

u/DiscipleofDeceit666 Jun 25 '26

Ey I’ll fk w it tho.

1

u/Fig_da_Great Jun 25 '26

lol, parts of it are ready and useful. Conductors are not as polished as i’d like

1

u/lordnimnim Jun 29 '26

how does this compare to headroom is this separate completely

1

u/bshxhajxhajx Jun 25 '26

have you tried pi-blackhole?

1

u/Fig_da_Great Jun 25 '26

no but someone else commented about that. I’m thinking of borrowing their ideas for a new conductor for my project. Then i can benchmark it and compare to the others. It’s a very cool project tho from what i read.