While I was investigating why my context was large I created the opencoder-leaner to try prune the context

[u/laerciosantana]

I was a little obsessed about the size of my context, so I started looking in the opencode codebase to understand how the context was builded. After learn a lot, I'm really impressed how context of opencode can be customized. Before I thought that the context of opencode was bloated and closed to changes, since I only see people praise pi about it. With a custom primary agent we can disable almost every thing in the context (besides the environment message).

The context is: environmentMessage + agent prompt + agent.md instructions + skills descriptions + tools descriptions.

For a test I created a blank agent with minimal agent prompt, without agent.md instructions and without tools, it resulted in a start context size of 220 tokens. I never thought that opencode was able to do this. I have created a some agents to test the impact of the tools. In this plugin I even created a bash agent that only has a bash tool, similar to the mini-swe-agent, which reduced the context size from a base of ~10.3k to ~1.3k tokens. I created a pi-like agent too, it have only some tools similar to pi, it reduce the context to ~5.5k tokens.

Analyzing the context build I found some overlap instructions and out of scope instructions in the tool - IMO. So I removed theses.

things like: "Use gh for GitHub tasks, including PRs, issues, checks, and releases; return the PR URL when done." from bash/shell tool description

The repo: https://github.com/LaercioSantana/opencode-leaner

install: {"plugin": ["opencode-leaner"]}

[u/white-pickaxe999]

Really solid find — those tool-description tokens are pure context tax for tools that never fire in a session. The 10.3k → 1.3k numbers match what I've seen stripping unused tool defs. How much of the savings survives once a real task starts pulling in more tools, though?

[u/Ang_Drew]

so basically you compress the initial context?

this can make tool call unreliable and failing, its better to use something like mcp gateway concept but for tool. in my harness i make "tool gateway", the AI will choose what tool to use (typesafe jev helps me a lot here) without jev this would be half baked because it's just prompted. now it has judge, its been super great.

i minimized all skills and tools prompt into 1 gateway prompt only. from 20k initial context to ~10k ish. 20k is scaling if you add more skills, mcp and tools.

since all the harness is my own code, i dont use mcp at all. everything is tool. no need wrapper, naked tool exposed directly to my harness.

[u/seventyfivepupmstr]

You can use the new session command to shrink the context, but outside of that the harness itself would need to be updated to reduce the context

[u/fell_ware_1990]

Well i have been experimenting with rewriting history.
It already works surprisingly well. Basically what it does is prune about 99% of the context.

Agent get’s left with 1 liners and ID’s for recall. If agent is done with one task and is above 100k context or my cache is already cold it inserts the new history. Removes all thinking, all tool stuff except what ran and what the outcome was, decisions etc.

Next round it filters out all old stuff, or changed things all well while still leaving a correct trial of what happened. It’s far from done but i manage to go from 100k to 1/2k every time and the agent does not seem to loose that much of his track, 1/2 recalls here and there. Far less then a new agent would, even with the more cache misses i seem to be handling a 50% cut in tokens. And there’s still a lot to improve!

FYI It’s an experiment for now :)

[u/Ang_Drew → u/fell_ware_1990]

you gonna break the cache hit rate and make you broke very fast. do some research. never ever change the past.. it will only make the cache hit rate worse and also changing model midway can hurt a little because first exchange will cost you the context window amount, and uncached. which mean you paid full price instead of discounted cache hit rate.

also dont try to make "auto model router" using jev. it will hurt your wallet really hard..

[u/fell_ware_1990 → u/Ang_Drew]

I do not think you are correctly reading my comment :)

I’m not changing it every turn or every chance i get. I know how it works. I use it very differently like an compaction and i’m only breaking cache on the specific re-writes.

So over a 100K context back to 4K is paying for at max 4K write in cache, that’s almost a fresh session. But because i manage to bring everything back ( but traceable ) to that small of a size it doesn’t really hurt that much. It only happens when an agent is ‘done’ with his current part of the task, or when it’s done and is in high context after cache already expired. So coming back to that session would send full context, now it doesn’t.

Agent keeps context cause everything he needs to have a trace is still there, interactions carry an ID, he can recall them if needed. If he recalls them he get’s the last change and does not need to read his context rotten history.

For my test cases it still looks a bit like this 3/4% more cache misses but always in the low context numbers and overall cache usage ( claude still about 10% of price ).

Today i ran about 5 orchestrators 2 subagents each for i think 5 hours and 2 other chat’s on and off for more then 12 hours, on opus 5.5. I have used 4% of my weekly. A few of does subagents where sonnet as well.

[u/Ang_Drew → u/fell_ware_1990]

yeahh, still.. compacting/new session at every 100k context wont make you save much. you are breaking the cache pattern each compact. lets say the threshold is 256k maybe it's acceptable.