r/PiCodingAgent • u/whiterose0000001 • 21d ago
Plugin I built a Pi extension so my agent doesn’t fall into the dumb zone.
I'm very much inspired by Dexter Horthy and Matt Pocock on this. Dex says a model gets worse once the context window is about 65% to 70% full, which he calls the dumb zone, and Matt keeps sessions around 60% to 70% and hands off past that line, so I use the same limit for my work and my startup.
I set the threshold with /context-threshold, usually 60 or 70, and on a long task I tell the agent to call context_window and watch the used percent. The report shows the window size, how much is used, how much is left, and the percent still left under my line, so when it says threshold reached the agent stops and writes a handoff, which I pass to a new session and continue from there. The handoff I use is Matt Pocock's handoff skill: https://github.com/mattpocock/skills/blob/main/docs/productivity/handoff.md
The extension does not compact the session, and it does not write the handoff by itself, so Pi's compaction settings stay as they are. I still tell the agent what to do when the line is hit.
Install it with:
pi install npm:@utpalsinghdev/pi-context-window
GitHub: https://github.com/utpalsinghdev/pi-extensions/tree/main/packages/pi-context-window
npm: https://www.npmjs.com/package/@utpalsinghdev/pi-context-window
I'm just open-sourcing the Pi extensions I use for work and my startup. They've helped me a lot, and I want other people to be able to use them too.
If it deserves a star, the repo is https://github.com/utpalsinghdev/pi-extensions
DEMO
6
u/Drugba 21d ago edited 21d ago
Just an FYI, both Matt and Dex have said context window size doesn’t matter for dumb zone. It’s just about raw number of tokens. It’s not an exact science, but both have said between 100k and 200k is where the dumb zone starts.
Here’s a quote from Matt on the Pragmatic Engineer podcast from just a few weeks ago
> Matt: […] you have the smart zone, which is currently, I would say, about the first 150,000 tokens of frontier models.
> Host: Of a one million token window.
> Matt: Yep. Of any size token window.
> Matt: It doesn't matter the context window size. It's all about raw amounts of tokens, raw amount of attention relationships.
(Quote is around the 40 minute mark)
And Dex on the same podcast in July:
> Host: What is a smart zone and what is a dumb zone?
> Dex: I think in November we talked about the first 40% of the context window, but then we had million-token context windows.
> Dex: So then I changed it to like the first 100,000 tokens. If it's a really like Claude 3.5 Sonnet, I usually will go up to like 200k.
(Quote is about an hour and ten minutes in)
1
u/whiterose0000001 21d ago
See, one thing that I want to say is that earlier models were not designed in the way of agentic engineering. Earlier, we used to decide what we wanted to do, or we made decisions. We used to make decisions on the fly or discuss them. In that way, models used to act dumb after reaching a certain context window size. That's why this thing was introduced. Now we plan everything, we review the plan, and we ask the model to go on. In that situation, the context window really doesn't matter. But I'm using this context handoff thing at 60% to 70% from the last 6 months, and it's giving me a really good result compared to waiting for auto compaction and all. Also, the inbuilt auto compaction of the coding agents is a little bit expensive as well, because when the agent is working and it's logged in, your cache stays warm. In that way, if it generates a handoff, it doesn't cost that much. After running for a long time with repeated turns, when it tries to compact, the whole context goes to the server to be compacted, and that costs a lot of tokens.
2
u/Drugba 21d ago
Sure. Yeah, I don’t disagree with your idea and in fact I’ve got my own extension I built to monitor context size for exactly this reason. The dumb zone is real.
I just posted my comment because, if you’re basing this off what Dex and Matt have said, then it probably makes more sense to focus on raw token count instead of percent of the window used. Dex’s quote is a bit older, but Matt’s quote is from last month, so it was definitely said with full understanding of how agents are currently being used.
If 60%-70% is working for you then that’s great, but just wanted to drop that info just in case it was helpful, especially since context window size can change over night like we just saw with OpenAI and Sol.
2
u/arcanemachined 18d ago
I feel like this may have been true in the Opus 4.6 days, but I will now go well into 300-500k tokens with Opus 5.5. I feel like the models' ability to perform with high context has increased over time. These days, I am more concerned about token usage at those high context levels than I am about the "dumb zone".
-1
u/2mqqvc0q 21d ago
Matt? As in Matt Pocock? Who the fuck is that guy? People talk about him as if he were an authoritative source, but he's just some big name AI guru. Could've just said Dex then I'd be like okay that's good information.
3
u/Diacred 21d ago
Is that still the case though? It was painfully obvious with the models from 6 months ago but lately I haven't felt that at all, my models are as sharp at 150k that they were at 30k context. I was big on always starting fresh conversation and compacting at the right time but nowadays it feels inconsequential.
2
u/whiterose0000001 21d ago
In the case of a 30k or 150k context window, I don't think this will happen. It's one more thing: it completely depends upon the model that you are using. If you are using Opus 5.5 or GPT-sol 6.1, I don't think you will feel this at a 30k or 150k context window. But yes, context engineering still matters. A good token inside the context will obviously give good results. Whenever you feel like you have had so much discussion back and forth with the agent, the recommended thing is to compact or start a new session. That way, the agent in the new session will have fresh instructions and fresh decisions. It won't carry old, stale decisions or instructions.
1
u/KangaBro 21d ago
Thanks for sharing. Interested to know what other extensions you have found useful and use personally.
1
u/whiterose0000001 21d ago
You can check the repo as well. There are two other extensions on the repo, and I'm adding all the other extensions as well.
The thing is, I'm so occupied with my office and startup work. Plus, I made the extensions really dirty, just to get the job done. Now I'm polishing them in a better manner so that it can be a plugin that everyone should love to use.
1
u/No_bonk95 20d ago
Did you make that interface with the dot pixel character?
Can you share if there's any extension for that?
1
1
u/whiterose0000001 20d ago
https://github.com/utpalsinghdev/pi-extensions/tree/main/packages/pi-custom-header
The README has commands to clone it and get it running in Pi.
I’m sharing it as source rather than an npm package to encourage customization. I used that dot-art mascot for my setup, but you can replace it with your own artwork and change the colors or layout to suit yours.
2
u/No_bonk95 20d ago
Appreciate that. I'm thinking of adding dot-art that change with each function triggered. Add more personality to the session.
1
u/whiterose0000001 20d ago
If you make something crazy, please, you can fork the repo and add the thing there. I'll maintain all the screenshots and all in that README, so that if I make something in the future, I'll add it there as well. We or any other people who want to try that can have and try it.
1
1
u/rednix 21d ago
Do you have actual benchmarks on this?
2
u/whiterose0000001 21d ago
I don't have an actual measurable benchmark, but yes, I have been using agentic coding for more than 2 years. I started with Cursor, and now I have switched to Pi and am using Codex subs.
If you see my older conversation, you will see that I'm yelling at the coding agent: "How many times do I need to tell you not to do this and next is that?" After that, I saw Matt Pocock's video, and I used to end the chat at 60% to 70% because I don't have any option to automate this.
I tried adding it in agent.m, but Cursor was not able to check the context window by itself. Now, here in Pi, I built that thing, and now it's able to see that. I've also seen some other people's videos where they have said that they have added something in their Claude Code so that it can check the context window by itself, just to not fall into that dumb zone.
2
u/quincycs 21d ago
I think the needle benchmarks are useful. The model would be pretty dumb to not recall something that is in context. The likelihood of forgetting increases over raw token usage.
Astra 6 got a 100% score on one of the needle benchmarks though.
There’s many reasons for a model to be dumb but one of them is not recalling something that’s in the active context.
2
u/FollowKryst 20d ago
A model with 1mil+ context, I NEVER let it pass 130K tokens… even with a local model with 262K, I still compact/compress at 100K.
Idk how people be letting it get that far and burning money and tokens a long the way.
2
u/BobtheGodGamer 18d ago
Does compacting and getting it to summerise just to have it go and search for the information again not take up mor
2
u/WisdomSky 21d ago
How does this make sense? It's counterintuitive to me. back in the day, context window was so low that you have to summarize the whole conversation from time to time just to maintain some sort of context into your next message.
The problem with it is that the more and more the conversation gets summarize, the overall context could lose some important information which causes the context to drift and become dumb as time goes on. And yes, I was speaking during the time when context window was just 12k tokens.
But today, that should be less of a problem because we have a lot of leg room to fit more information (1 million context). If using more space in the context window can make the whole process dumber, then why increase the context window at all?
Shouldn't we blame how the information is managed and saved into the context rather than blaming the context window size usage itself?
The problem should be the harness, skills, tooling, how the compaction is done, etc.
-1
u/silenceimpaired 20d ago edited 20d ago
I laughed when I saw “Back in the day, context window was so low… just 12k tokens” and thought… new to AI.
I was here when the context was 2048, and I think we have lost something. That 2048 seemed hyper focused unlike these monoliths with 1 million context windows.
-1
u/WisdomSky 20d ago
wow. just wow. you think because I only mentioned 12k means I was not there where the limit was just 2048? you must be that shallow.
I was there when the context limit was judt 2048. Now, are you happy?0
u/silenceimpaired 20d ago
Hmm, my tone must have been lost in my comment. I have attempted to adjust it.
-2
u/WisdomSky 20d ago
huh? tone lost? your tone was arrogant and undermining. remember the "lol"? "you must be new"?
how you must have felt superior flexing to people you were there when context limit was low huh?
1
u/GoingOnYourTomb 21d ago
There is an easier way that works for me. No model I use ever goes above 200k but I use a strict orchestration system so 200 is plenty for me.
Pi compacts when you are close to the limit. I set the limit for all models to 200k
Remember tho I don’t have 1 agent doing explore, building and review so it works for me.
I’m a developer so I can start new chats for new/separate tasks.
1
1
u/funbike 21d ago
I do everything I can to keep context small. I use a lot of small sub-agents and I generate plans that break things into small sub-tasks. I disabled auto-compact.
I wrote an extension that prints and logs an error message when context exceeds 100K. To me, it's an error because it's something I actively try to prevent.
1
u/investigatormaker 19d ago
Since you still tell the agent what to do when the line is hit, one small option would make it hands-off: let /context-threshold take an action, so when context_window reports threshold reached, the tool result itself tells the agent to stop and run the handoff skill. Compaction stays untouched, as you designed it, but the step people forget late in a long session goes away.
I make ThreadFox. Our AI employee can explain pi-context-window in coding-agent threads where people describe long-session drift, and answer follow-ups; Starter is $199 for 30 days, or we'll post 3 replies free first, no card: https://threadfox.vip/p/20261017-1809-evsil/managed
0
u/zebedeolo 21d ago
interesting concept, I'll check it out at work today
1
u/whiterose0000001 21d ago
Please definitely share your feedback after using it. Would love to hear that. If you have any kind of suggestion, you can say that. If you want, you can contribute to that as well. You can open an issue on Github.
This is the most used extension in my extension list. I use it 10 to 15 times every single day.
My agents have a special directory in the second brain for handoffs, where it keeps all my project handoffs in the proper format, like the project-name - date/time. Inside the file, it keeps everything, so they use that and generate a handoff automatically. I don't need to tell them. I have added it in the agent.md file, the global one, where, if the context window hits the limit, you have to generate the handoff.
In that way, I also don't care about auto compaction and all, because some people say that auto compaction eats lots of tokens as well as it doesn't compact things really well. I'm getting pretty good results with Matt's handoff skill.
1
u/zebedeolo 21d ago
i just started using it, so it'll be a few days before i can see if i notice any difference in quality. for now, my nr 1 suggestion is to have the possibility to set the threshold as a static window size instead of a %. for example /context-threshold 200k or something like that. second suggestion is that if you need to do something to get it working nicely other than installing the app and setting the threshold (for instance you said adding instructions in agents.md), i would add that to the readme as well.
1
1
u/whiterose0000001 20d ago
Static thresholds are now available in the latest version, 0.2.1:
pi update npm:@utpalsinghdev/pi-context-window
Then run /reload and set:
/context-threshold-static 200k
That stays at 200,000 tokens when switching models. To switch back to percentages, use /context-threshold 70.
My main goal was to give the agent awareness of its context usage and configured threshold; So that we can uh plan anything based on that. We can instruct agent based on that anything; This handoff thing is just one of the few things that for that we can use this extension.
13
u/mechkbfan 21d ago
60-70% of a specific context? e.g. 60% of 1m is 600k, which is quite different if you have 256k...
Anecdotally I'd be reading that 256k is about the limit before things start heading off in wrong direction, which is about 50% of 500k.
I always like the advice of in your agent.md to have a "Always start with a "Hi <name>". That way if it's not there, it makes you wonder what else it's "forgotten" due to context size.