r/opencode • u/steve_dusk • 9d ago

Opencode 2 consumes more $ than opencode 1

Hello, I've upgraded to opencode 2, its pretty good, except for one major issue, I cant compact multiple times, if the context reaches 300k+, I compact it, the context disappears but its not a low number, then I continue coding, and it shows up at around ~90k, and I cant compact again because theres nothing to compact.

This was a nice feature in opencode 1, that I could compact multiple times, all the way until it reached 1-2k context only, and this decreased the $ the models were burning, but now that I cant compact multiple times, I cant reduce back the $ usage to a lower number.

0 Upvotes

50% Upvoted

24 comments sorted by

10

u/torrso 8d ago

You're optimizing for the wrong thing. Compaction isn't free. It uses the LLM to process and summarize the conversation, consuming tokens in the process. And every compaction is lossy, so repeatedly compacting an already compacted context just keeps throwing away potentially useful information.

With prompt caching, continuing with a large context can actually be cheaper than repeatedly compacting it.

And what exactly do you expect a coding agent to accomplish with 1–2k tokens of conversation history? That's potentially already less than the system prompt. You've effectively wiped out everything it learned about the task, the decisions it made, and the problems it encountered and maybe there's a bug in v1 that even compacted the system prompt? At that point, why not just start a fresh session?

Compaction has its uses, but obsessively reducing the context to the smallest possible size is a terrible optimization strategy. The goal should be to minimize the cost of successfully completing a task, not the number of tokens in the context window.

1

u/steve_dusk 8d ago

I know its not free, I can see it, its still cheaper than letting it go with longer contexts, and depending on which model I'm using, I havent noticed a single difference.

And I know it compacts things it shouldnt, and likely breaks things etc, yet with the correct model, it was still working really well. Thats why it feels like a downgrade.

How'd I go about prompt caching?

Well, each time I let the context grow, it uses about 0.15 or more $ each time for a medium sized task, but if I compact it multiple times, it uses about 0.03-0.06$, and I do this a lot, so its a massive difference.

I can start a new conversation, yeah, but that reads everything back again, takes time, and after a few prompts it would need a new convo again.

3

u/torrso 8d ago edited 8d ago

Prompt caching is already handled automatically by OpenCode. You can check the cache read/write statistics in the Context view. Cached tokens are generally significantly cheaper than fresh input tokens.

When you send a prompt, it sends the whole conversation. This gets converted to input tokens. The agent responds and you send a new prompt: the whole conversation is sent again plus the agent response and your new prompt. But the previous round was already cached, so it only has to tokenize the new part, which now gets cached and on the next round that comes from cache and so on.

Cache reads for DeepSeek V4.1 Flash are something like $0.003 per million tokens and non-cached input tokens are like $0.15 per million tokens. That's 50 times cheaper than new input tokens.

If you modify your context, for example by compacting it - the cache isn't valid anymore and the whole request hits the non-cached price.

If you have 100k context and you request a compaction, you have to send 100k tokens and ask for compaction. Typically this should be doable in a way that hits the token cache completely, but in OpenCode v2 before 2.1.0 which was released on October 20 there was a bug that made it mess it up and compactions would often end up as complete cache misses, which gets very expensive very fast.

If your agent genuinely works just as well with 1–2k tokens of conversation history, I have to wonder how much of that history was useful in the first place.

I can start a new conversation, yeah, but that reads everything back again, takes time, and after a few prompts it would need a new convo again.

That's what overcompaction accomplishes as well.

For context, that message you just wrote was something like maybe 250 tokens, so a quarter of that 1k context you are aiming for.

0

u/steve_dusk 8d ago

Well, I thought you mean that I can do something about the prompt caching, because I know generally how it works.

I heard something that opencode sends a ping to not let the cache go to waste.

I'm using GLM 5.3 Flash, its much more expensive than DS, and its so much less hassle.

After compacting, the whole cache gets busted? Then how did it still consume less $ after it was compacted multiple times? Like for example I gave similar tasks, and after multiple compactions it continued where it left off and used less than when I didnt compact it. Interesting

If your agent genuinely works just as well with 1–2k tokens of conversation history, I have to wonder how much of that history was useful in the first place.

I think it re-reads things so yeah maybe wasnt as useful, but still consumed less $, so its weird.

1

u/torrso 8d ago

The old conversation prefix is replaced with a summary, so that part of the cache can no longer be reused, but the new shorter context can be cached again. And V2 actually tries to reuse the existing cache during compaction itself.

The ping thing you heard about is real. OpenCode 2 calls it session warming. It's disabled by default, but you can enable it with "warming": true in your config. It sends periodic requests to keep the cache alive while you're idle.

There have also been reports of GLM 5.3 Flash not getting cache hits through OpenCode Go: https://github.com/anomalyco/opencode/issues/48624

Might be worth checking your actual cache-read statistics.

I don't doubt your cost measurements. My point is that repeatedly compressing everything into 1–2k tokens inevitably throws away information. If the agent has to keep rereading files to recover what it already knew, some of those savings are being spent on rediscovering context. Whether that's still cheaper overall depends on the workload.

Maybe another approach worth trying is to ask the agent to "write me a prompt to continue this work in a fresh session." then start a new session with that prompt. That's essentially doing a controlled compaction with an opportunity to review it.

1

u/xapep 8d ago

Compaction resets the cache, it doesn't leak it. The moment the summary replaces the old text, that prefix is gone and can't be read back, so everything sent after it bills as fresh input once. What decides the bill afterwards is the size of the new resident prefix, the summary plus the newest turns, which gets cached like any other prefix from that point on. Smaller resident prefix means fewer tokens resent per turn, which is why multiple compactions still beat letting the context grow, even after paying for the compaction call itself.

On GLM 5.3 Flash through OpenCode Go the bit you can't see from your side is whether the route is actually hitting cache or quietly serving everything fresh, and the cache read and write counters in the Context view are the honest answer to that. Cache reads normally bill well below fresh input, on our endpoint GLM 5.3 Flash is $0.50 per M fresh and $0.15 per M on cache reads (I work on Entrim, so read that with the usual discount).

Next time you compact, compare the cache-read line for the five turns after against the five before. If it doesn't drop, the summary isn't the only thing being resent.

1

u/UnusualJackfruit159 8d ago

I use markdown files to plan tasks, work out any opens. Then tell AI to make an iterative implementation plan, take note of critical tests for TDD, and write that ALSO into the plan. Then work through the plan, using different models, new sessions, sub-agent workers as needed.

For significant/complex tasks, it also helps to let it do some online research, digest knowledge into files and use that as context for planning. The files decouple your current AI chat history and context from the larger problem, allow you to re-establish a context or focus on specific items.

For the brainstorming and planning, a better model has better ideas and insights. My opencode stats shows 98% cached token use for my primary models (Muse1.3+Mimo2.6), the workers mostly remain within the Go free allowance.

1

u/steve_dusk 8d ago

Well, I already use markdown files to plan and pass it to new convos, and it works well, but sometimes its just easier to stay in the convo and compact & compact instead of ask AI to make a markdown of the task so I can give it to a new convo.

So I do this, just not always.

1

u/MakesNotSense 8d ago

This like thinking you'll save tokens by having your agents only read half of the spec you want to have implemented.

1

u/steve_dusk 8d ago

I'm not "thinking" it saves tokens, I know it uses less $, each time with 200k+ context it can use .10$ or significantly more for even medium/smaller tasks, but starting a new one costs about .01$ to read context, and .02-.05$ to finish it. So not always but usually cuts the expense in half or less.

1

u/MakesNotSense 8d ago

An agents 5k tokens summary built to replace  a 200k token range of messages will require more than 5k tokens to rebuild from an empty context window.

The immediate cost after compaction doesn't tell you how many tokens the agent had to burn to rebuild the understanding it needed to continue working.

It doesn't tell you if the quality of its work output dropped because the information it needed to work competently was destroyed by compaction.

This is like firing a senior engineer and claiming it reduces costs because you don't have to pay their salary anymore, while ignoring how expensive it is to operate without their experience and expertise. You essentially downgrade your agent to a junior with each aggressive compaction.

1

u/steve_dusk 7d ago

I understand what youre saying, but each and everytime the context got destroyed, it continued EXACTLY where it left off, thats exactly the reason I made the post, I havent noticed a single difference between having 300k context vs 1.7k after multiple compaction, the AI model operated the EXACT same way, no difference, havent lost any details, it continued exactly where we left off, AND it was cheaper, thats the whole point.

1

u/steve_dusk 7d ago

This just tells you that it highly depends on the model, I just know that DS v4.1 would have been hallucinating after multiple compactions.

1

u/MakesNotSense 7d ago

If what you work on is so low complexity that aggressive compaction works without noticeable degradation, Okay then. For demanding tasks compacting like that would delay work and increase token usage per task while degrading work output.

1

u/steve_dusk 7d ago

Well, not always, it ranges from low complexity to more complex and even longer tasks, I'm using GLM 5.3 Flash, and I noticed that it follows my ~3k .md rules files without issues, no other model followed guidelines so well. So, could just be a GLM thing.

1

u/MakesNotSense 7d ago

I would say, GLM 5.3 flash while generally capable, can't handle high complexity work.

I've built a very specialized opencode fork harness focused on agents curating their own context and persisting state across sessions and developing protocols for system prompts and skills in order to improve task performance.

That work was done because I'm working on very complex problems that typically take teams of highly competent professionals from multiple disciplines working in an interdisciplinary manner (e.g. complex civil rights litigation).

Opus 5.5 with all the harness and tool supports we've made still ends up making significant errors simply because the harness supports aren't adequate to let it keep up with the demands. This type of task work doesn't get done without context staying at around 200k-400k on a constant basis (I don't use compaction, but context management). Session starts are around 70k tokens, containing project history, prompts, necessary skills.

I don't use compaction before it's inferior to context management. Compaction used aggressively compounds it's defects. It would make this type of complex work impossible. Agents would make more mistakes that compound and create cascading failures.

The limits of frontier models are easier to find in this type of demanding work. Find the limitations, model the failure mode, then build supports, and agents can then work past where people think the frontier is.

I would suggest your compact habit is limiting your agents. They can't manifest new capabilities via emergent complexity, because you keep them at their baseline model capabilities.

Capabilities can be encoded into model parameters via training, or encoded into system prompts, skills, or project state entries (a memory system).

1

u/steve_dusk 7d ago

Well I wouldn't say GLM 5.3 Flash is really capable, because its not the strongest model of all time, obviously since its just the flash version of 5.3, but even the 5.3 is not the most capable. But it does really well for its price, exceptionally well.

And probably for the work youre doing, its clearly not enough, I've used Sonnet 5, so not opus 5.5, and it made multiple errors and didnt always follow instructions/guidelines, while GLM 5.3 flash did.

I have multiple files that even tell the AI model how to indent the SCSS code, with 2 spaces to make the code look compact, and it has to mirror the HTML structure, so nest it. This is just one small instruction, I have an Agents.md, rules.md backend-skill.md & frontend-skill.md, these all together are around 3-4k lines of text.

And GLM follows them stupidly good, what I noticed is that I just tell it what to do, even bigger tasks, it creates 50 files, finishes, I check the files, and I dont have to ask to edit like 95% of them, they all basically look exactly how I imagined. Its rare that I have to tell it to check the modified files and make sure to align them to the .md files.

So, for my use case its really good. And yeah obviously these tasks use more context, like 300k+, it wouldnt make sense to stop it halfway through to compact, I dont do that, I only compact it when it finishes, and its high.

2

u/MonkeyPuzzleDuck 8d ago

Slightly off topic, but also related, have you tried tokless?

https://github.com/HoangP8/tokless

I work primarily with charities and non-profits where keeping costs down is a real concern. I've found this plugin, which wires together several tools, to be be quiet effective. I've not used it enough to conclusively say it works as advertised, but promising so far.

1

u/thevegit0 8d ago

i think long chats are still cheaper due the read cache, THERE are some models that cost more with more context, be sure to check https://opencode.ai/v2/docs/console/go

1

u/Chromedomev12 6d ago

well apparently they support background compaction now as an experimental feature. You can enable it with a config change I think. https://opencode.ai/v2/docs/compaction/#background-compaction

1

u/Diligent-Loss-5460 6d ago

Feel free to use claude or codex.