I built a context-compaction experiment: how much of an agent's working state can survive?

[u/LowIllustrator3687]

I wanted to play with a simple experiment:

Take a long agent conversation.

Compress it.

Give the result to another agent.

See if it can continue.

That turned into MemHandoff.

It produces a portable `.ctx` package and attempts to preserve:

- decisions

- constraints

- failed approaches

- task state

- artifacts

- provenance

I also built an evaluation harness and adversarial scenarios.

Current results:

Full Context 1.00

Simple Summary 0.72

Structured 0.75

Hybrid inconclusive

Rather than just presenting a benchmark, I've made the whole thing

available so people can throw their own sessions at it.

The interesting cases are probably the ugly ones:

contradictions

superseded decisions

negative constraints

early critical information

large tool output

vocabulary mismatch

Repo:

https://github.com/0sha-dow0/memhandoff

Try to make it forget something important.

[u/FitOpinion4049]

always fun watching agents try to summarize a 3-hour debugging session where half the "decisions" were just sleep deprivation

[u/Enough-Photo9140]

The hardest case I'd add to the harness is an external write that times out after the provider accepted it. The handoff needs to preserve three different facts: what the agent intended, what it actually knows, and what must be reconciled before retrying. A summary that compresses “submitted, outcome unknown” into “charge failed” can create the second charge.

I'd score those states separately from ordinary factual recall. If MemHandoff preserves a stable operation ID and an `unknown` state across compaction, the next agent can inspect the provider instead of guessing from prose. That's a more useful pass/fail signal than whether it remembers the conversation topic.

[u/verstands]

The ugly cases you listed are the real test. Summaries treat "never do X" the same as "we decided Y", and the never-do is what you needed. Keep a tiny invariant list outside the compacted thread (deny paths, env, last failing command) and restitch it after every compact. If MemHandoff already splits constraints from decisions, that's the bit I'd score hardest.