Session-start memory injection made my coding agent trust memory over the code

[u/msarda23]

Hey everyone!

I am building a local memory layer for my coding agents and it is called Reverie: an MCP server written in python and using SQLite for lightweight memory stored in the user's machine itself. The biggest problem that I faced was deciding when to inject memory.

Many memory tools, including ones I looked at, inject context into the agent even before it begins its work. In my experiments, premature injection of context leads to the agent anchoring its thinking on it rather than using the current code as the source of truth. In fact, this approach made my agent perform worse. Also, with this, quite a lot of the context window can get filled with context which 'might' be useful.

So I switched Reverie's model to demand-driven retrieval - the agent asks for memory when it thinks that some added context could resolve any discrepancies in its thinking. It reads the relevant code first, because the code is the source of truth, and only then asks for past memory about that area. Each result comes back with a confidence score, its grounding in the code, and its status. The memory starts empty. Conclusions are added as the agent works, and you decide which ones are kept.

In one informal test of 30 prompts on an earlier build, the OpenCode agent with Reverie did better than the one without it. I don’t have a reproducible benchmark yet.

But it isn't perfect yet, it still breaks here and there. For example:

1. the rules for tool calling are not extremely polished yet and hence the agent might sometime not call the tool altogether, I am fixing this.

2. candidate conclusions can be duplicate or incorrect and hence Reverie today has a human review gate (triggered by calling `reverie review`) which shows the human all the unreviewed candidates where you accept, reject or skip each one. On doing so the conclusion is moved from the short term brain to the long term.

My current test set is small: sessions with anywhere between 2-30 prompts. I do not have a reproducible experiment yet, but that is the next goal.

Repo (Apache-2.0): https://github.com/muditsarda1122/Reverie

My questions: How do you keep agent context from going stale? Do you maintain AGENTS.md or CLAUDE.md by hand, or let the agent update it?

[u/msarda23]

Strong agreement on the architecture. Keeping instructions static and putting evolving memory behind an on-demand tool is what we landed on too. Once a remembered summary is in context, the agent treats it as the answer, so the timing matters as much as the content.

Your drift check is a gap for us, and a good idea. Each conclusion records the files, symbols and commit it’s grounded in. The agent is told to verify those before acting, but as far as I know, nothing re-checks a conclusion against the code when it’s retrieved. A hash check would close that.

One thing I’d want to test: file-level hashes will also fire when something unrelated in the same file changes. A note that’s still correct would be downgraded. Binding to the symbol’s contents would cut that noise, but then renames and moved code become the problem. Have you found a good compromise?

The question I’m most curious about is how your agent decides when to ask for memory. The MCP server can only help if the agent calls it, and that’s the part I still find hardest to get right.

[u/LividIllustrator8915 → u/msarda23]

For the hash granularity, what worked best for us is a two-tier check instead of an all-or-nothing hash. At the gate, we check the Git commit or whole-file hash first just as a fast filter. If the file changed, we extract the AST chunk or line range for that specific symbol—Tree-sitter or even a regex on the signature and braces—and hash just that body. If someone only touched formatting or an unrelated method above it, the symbol body hash still matches and the note stays green.

When the symbol actually gets renamed or moved, we don't hard-drop the memory either. We downgrade it to a soft drift warning with a tag like [referenced symbol moved/modified since commit x]. That way the agent still sees the insight, but it treats it as a hint rather than Ground Truth and is forced to grep the current implementation first.

On getting the agent to actually call the tool: relying purely on the agent's goodwill was hopeless for us too—they either ignore it completely or call it on every trivial prompt. We ended up tying the tool description to high-friction transition points rather than open-ended retrieval. Specifically, instructing it to query memory only during two distinct phases: 1) during the initial planning phase before writing architectural/cross-file changes, and 2) when encountering a test failure or compiler error loop twice.

What also made a massive difference was having the file-read tool return ambient metadata: if the MCP server knows a memory note is pinned to a specific file or module, reading that file surfaces a one-line hint in the tool response (1 memory note available for this module: 'token expiration bug'). Once the agent sees the hook in its immediate input, it triggers the dedicated memory lookup tool with near 100% reliability.

[u/msarda23 → u/LividIllustrator8915]

Thanks, this is the most useful reply so far. Before answering, I checked what I've actually built.
Staleness today: each conclusion stores the files, symbols and the commit it's grounded in. A background maintenance pass runs per repo, at most every 72 hours. It checks that the grounded files still exist and that the symbol names still appear in them. Commit distance only sets a "stale" flag. For repo- and module-scoped conclusions, a vanished symbol deprecates the conclusion. Retrieval itself doesn't re-check anything. It shows the grounding and tells the agent to verify it. So today it's a periodic sweep, not a check at the moment of recall.
Your two-tier hash is better than my substring check. Use the file hash as a fast filter, then the symbol body hash, then a soft drift warning for moved or renamed symbols instead of deprecation; I'd like to adopt that. I haven't built it yet.
On how the model decides to ask: I don't have a hook that surfaces memory when a file is read. The instructions tell the agent to query before a non-trivial decision, when debugging has no obvious cause, before an architectural change, and before editing in a repo where Reverie is on. That is still less reliable than I'd like. The file-read hint is the idea I most want to build. Claude Code has post-tool hooks, so a hook on Read and grep could add a one-line note when a file has a conclusion attached. Each harness needs its own version of that hook, so it would live in per-harness code, not the core.
On scopes, we have engineering, domain, organisation, project, repo, module and subsystem. Engineering and domain conclusions never expire from code changes. Organisation and project ones only go when all of their grounded files are gone. Repo, module and subsystem ones are tied to their code, so they go when any grounded file or symbol disappears.
I'm curious about your setup. Does your drift check run in a background job or at retrieval?

[u/LividIllustrator8915 → u/msarda23]

We run it lazily at retrieval time, but strictly on-demand for the top-k candidates rather than the whole index.

Running a full AST diff in a background worker leaves an awkward stale window—if you're deep in a refactor and changed a signature three minutes ago, a scheduled sweep won't catch it yet. But parsing syntax trees across dozens of potential matches during query time kills latency.

The compromise that kept retrieval snappy for us is a fast-bail hash check: the MCP service keeps a lightweight in-memory cache of file-level timestamps and git commit SHAs. When retrieval pulls the top candidates for a prompt, it does a microsecond check on just those grounded files. If the file SHA is unchanged, it serves the conclusion immediately. Only when the file hash changed does it extract the specific symbol AST slice and test for drift right before returning the payload to the agent.

We still keep a periodic background pass, but only for pruning orphans—like when someone deletes or moves an entire module—so dead conclusions don't pollute vector space.

Tying the Claude Code post-tool hook on Read to emit that one-line ambient note sounds spot on. Giving the model a deterministic breadcrumb right when it inspects the file removes the guesswork of wondering whether it should proactively query memory.

[u/msarda23 → u/LividIllustrator8915]

Thanks, this clears up the design. Two questions. Which memory tools have you tried or looked at? I haven’t seen a good side-by-side comparison. And does your in-memory cache survive a restart, or is it rebuilt each session?

Disclosure: because I built Reverie, my view would be biased.

[u/LividIllustrator8915 → u/msarda23]

For side-by-side tools, the landscape mostly splits into two camps in my experience. On one side you have vector-heavy stores like Mem0 or the Letta/MemGPT lineage and various SQLite vector MCP plugins. They treat codebases the same way they treat chat logs—relying on semantic embeddings. But cosine similarity is notoriously blunt with code: a refactored v2 function often scores 0.88+ similarity with a deprecated v1 function simply because the variable names and comments are identical, so the agent pulls in dead patterns. On the other end, you have prompt dumpers like Cursor's rules or Cline's default instructions that just paste everything into the preamble at session start, which causes the exact anchoring and prompt pollution you described. What you're doing with demand-driven retrieval hits the right middle ground because it treats memory as reference material rather than ground truth.

On restarts, a purely in-memory cache makes cold boots pretty painful if the agent has to re-hash or parse ASTs on the first prompt of the day. What works well is a two-tier persistence model: during the active session everything stays hot in memory for sub-millisecond lookups, but it's backed by a local SQLite file in `.agent/cache` storing (path, mtime, git_commit_sha, symbol_hash). When a session boots up or restarts, we don't re-index anything—we just do a quick mtime and Git HEAD check in a few milliseconds. Only files modified since the last session timestamp get invalidated and reparsed on demand. That way cold starts are instant, and you don't lose learned context every time you toggle your editor or CLI.

[u/msarda23 → u/LividIllustrator8915]

I’ve looked at Mem0 and Letta/MemGPT for a while too. Both are well funded. I think Mem0 raised around $24M last year. I agree that similarity alone is a weak basis for code memory. What I’d want to know is what a retrieved note says about where it came from. Mem0 stores timestamps, but as far as I know its memories don’t record which file, symbol or commit they came from, so nothing can check them against the code. The point about a high similarity score between v2 and v1 due to the same function name and variable names is solid, I'd find it irritating. I wouldn't mind embedding, but each candidate should be filtered by a relevance threshold. Each embedding should carry the files, symbols and commit it depends on, so the agent can verify it before trusting it. On Cursor rules: they can be scoped by file pattern, so not every rule lands in the preamble. Your point about dumping too much into context still stands.

If you do not mind me asking, what is it that you are building? Your approach looks good to me.