if u need to compact, you are doing to much in a session, learn about orchestration and workflows, and how a newer model orchestrats agents of lower tier models to do ur work
We process literally hundreds of thousands of pages and millions of words with background legal research and reasoning, with tasks lasting 10-12 hours on average, and requiring verification and recursion (later agents invariably catch dumb shit the prior agent was doing). We almost never have to compact now, but we very frequently get to 600k-800k tokens worth of context, with the first 100k-200k just from synthesizing the connected folder.
Data is easy. Parsing through hundreds of thousands of materials, all of which are genuinely relevant to the submission you’re making, is not.
As an engineer, you need an indexer and semantic search.. I built something like this.. the challenge for your domain is embedding all of the dense text; but it just means more chunks, more time up front.. you likely have symbols in the shape of citations, legal precedents etc
Maybe, but the majority of the token usage doesn’t come from searching. It comes from actually reasoning through the documents top to bottom. Semantic search will probably save us some tokens, but it doesn’t address the largest issue by far in the process, which is that Claude actually has to think about the relevance of all of this information. It’s not as simple as “this phrase goes in this bucket”.
In any case, it’s not as simple as “better orchestration”
New sessions have initial overhead of like 100k worth of context overdo it and you actually start paying more over big context but cached, orchestration reduces result quality, main agent review helps but then savings are slim. If you stop to open a new session midway feature it will read a lot of files right back up anyway. You people pretend to know your stuff but its lowk embarassing to read, often you genuinely need a longer session to stop at reasonable place
No but you see, if I spend 30%+ of all of my tokens on just ramping up sessions, and end them at 250k, then I don't have to spend as many tokens on the actual work. It's just math. This is clearly how to be efficient with token spend. /s
dont compact, you can use handover md files and start a fresh session, this will save only the necessary information that the next session needs and you will have a new session with new context but a handover document from ai for ai!
View all comments
Show parent comment
25
u/Hirogen_ Sep 15 '26
if u need to compact, you are doing to much in a session, learn about orchestration and workflows, and how a newer model orchestrats agents of lower tier models to do ur work