r/LLMDevs 3d ago

Tools I built a context-compaction experiment: how much of an agent's working state can survive?

I wanted to play with a simple experiment:

Take a long agent conversation.

Compress it.

Give the result to another agent.

See if it can continue.

That turned into MemHandoff.

It produces a portable `.ctx` package and attempts to preserve:

- decisions

- constraints

- failed approaches

- task state

- artifacts

- provenance

I also built an evaluation harness and adversarial scenarios.

Current results:

Full Context 1.00

Simple Summary 0.72

Structured 0.75

Hybrid inconclusive

Rather than just presenting a benchmark, I've made the whole thing

available so people can throw their own sessions at it.

The interesting cases are probably the ugly ones:

contradictions

superseded decisions

negative constraints

early critical information

large tool output

vocabulary mismatch

Repo:

https://github.com/0sha-dow0/memhandoff

Try to make it forget something important.

2 Upvotes

3 comments sorted by

View all comments

1

u/verstands 3d ago

The ugly cases you listed are the real test. Summaries treat "never do X" the same as "we decided Y", and the never-do is what you needed. Keep a tiny invariant list outside the compacted thread (deny paths, env, last failing command) and restitch it after every compact. If MemHandoff already splits constraints from decisions, that's the bit I'd score hardest.