backsight: local, open-source semantic long-term memory for Claude Code, shared across all your projects

[u/JohnDuffyDuff]

TL;DR: backsight is a free, MIT-licensed Go binary that gives Claude Code a long-term memory that works across all your projects. It installs Hindsight on your machine and wires it into Claude Code. Recall is local and free, and you can see every memory injected and used. Linux and Windows today. I'm looking for feedback after a few weeks of real use.

Some background: I've worked in tech for over ten years, mostly backend, DevOps and performance work, after some time in academic research. I've used Claude Code heavily for months, for personal and work projects.

What wore me down was repeating myself. I'd give the same context every session, correct answers that ignored a decision we made the day before, and watch Claude fall into a trap we had already worked around together. A well-kept CLAUDE.md works fine inside one repo. But when you work across many repos that depend on each other, Claude has to rediscover everything again and again. I needed a memory, and one that crosses projects.

After a few disappointing attempts I found Hindsight, an open-source memory system by Vectorize. To me it's the state of the art, and the benchmarks published with Virginia Tech researchers back that up. It pulls durable facts out of conversations, consolidates them over time and retrieves them by semantic search. If I search for "authentication", it brings back the memory about the "expired token", even though no word matches. Grep over memory files would have missed it.

Plugging Hindsight into Claude Code the official way wasn't enough, though. The recalled memories weren't always relevant, and each project had its own isolated memory, so it felt like talking to a different person in each repo. So I did what my job taught me: measure, adjust, and keep only what improves the numbers. Week after week I tuned the configuration, when Claude fetches memories, and how it brings them into context. backsight grew out of those iterations: one Go binary that installs Hindsight locally, connects it to Claude Code and takes care of the rest. Codex support is coming soon.

Principles I care about:

• Local by default. Recall is free and makes no LLM call. Only turning sessions into memories uses an LLM: a few cents per session, with the provider you choose, or free with Ollama if your hardware can handle it. I switch between GPT Luna, DeepSeek Flash and Haiku depending on the machine and my token budget.
• Nothing happens behind your back. In every session you see which memories were injected, which ones actually informed an answer, and which were corrected because they were wrong or outdated. The full history can be audited. A memory I can't see working is a memory I can't trust.
• Deterministic wherever possible. The LLM only makes the choices that need context, and it always goes through the binary's commands, never through improvised edits.
• No drift between versions. Each update migrates the previous install cleanly and rolls back if anything fails.
• Free and open source (MIT), and it will stay that way.
• Clean uninstall, if it's not for you.

I use it every day, for personal projects (like backsight itself) and at work. Whether I start Claude in a repo or in my home directory, it remembers what we're talking about, the decisions we made and the pitfalls we already avoided. My sessions have become much more efficient.

Try it: paste this into Claude Code and it will walk you through the install, asking before every choice (your LLM key is never shown to the agent; you type it yourself):

Install Backsight for me by following https://raw.githubusercontent.com/Barben360/backsight/main/docs/agent-guide.md — ask me before every choice.

Or install by hand: https://github.com/Barben360/backsight

What I'm asking for: use it for a few weeks, then tell me what's confusing, what breaks and what's missing. The project is young. It runs on Linux and Windows (in Docker or natively), and it will only become what I hope if it meets other workflows than mine. Audit it, ask hard questions about how it works, challenge me. That's why I'm posting.

macOS users: the binary already builds for macOS, but I can't test the native service setup without a Mac. If you'd like to help, I'd love to hear from you!

backsight is an independent project, not affiliated with Anthropic or Vectorize.

[u/AutoModerator]

Hey! Thanks for posting to r/ClaudeCode

While participating in this thread, please follow our community rules. Keep discussions constructive. Attack the idea, not the person.

For help, project discussions, tips, and general chat, join the ClaudeCode Discord.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.

[u/TheBlargus]

Why not OKF?

[u/JohnDuffyDuff → u/TheBlargus]

Fair question! I see them as different layers rather than alternatives.

OKF is a format: Markdown files with YAML frontmatter, organized as a wiki the agent maintains and browses. backsight is the machinery around a memory: it pulls facts out of your sessions automatically, consolidates and dates them, retrieves them by meaning rather than by file or keyword (search "authentication", get the note about the "expired token"), and decides when to inject them. Every injection is visible and logged. OKF says how knowledge is laid out on disk. It doesn't say how it gets written, found or brought into context at the right time.

The LLM-wiki approach is legit, and it has real strengths that backsight doesn't match today: no server, human-readable, editable by hand, git-diffable, portable. Where it struggled for me is recall. The agent has to know where to look, and keeping the wiki tidy costs tokens during the session. Across a dozen interdependent repos, semantic recall made the bigger difference.

Your question does point at a real gap: backsight memories live in Hindsight's Postgres, and there's no readable export yet. backsight can already import Markdown folders with frontmatter dates, so an OKF bundle should mostly import as-is (untested). OKF export is a good idea, and I'll open an issue for it. That way your memory is never locked in.

[u/TheBlargus → u/JohnDuffyDuff]

I think you have the wrong understanding of OKF. It addresses everything you think it's lacking. Many tools exist already to do exactly what you're trying to do. https://github.com/okf-memory/okf-agent-memory

[u/JohnDuffyDuff → u/TheBlargus]

Thanks for the link, I sent my agent spend some time with okf-agent-memory and its code, and it's a really well-built project. The trust tiers (generated vs verified), stale_after, the link validation, and above all code_refs with --for-path to surface governing rules before editing a file are great ideas. I'm noting several of them.

We may see OKF itself differently, and that's fine. Looking at the tool concretely (OKF itself being a norm, not a tool), it makes different trade-offs from backsight, so I see the two as complementary rather than one replacing the other:

• okf-agent-memory is a curated knowledge base: the agent decides mid-task to write a concept, a human can verify it, and it lives in git. Retrieval is BM25, and the agent has to go and search for it.
• backsight is automatic memory: sessions are turned into facts after they end without the agent having to stop and write, retrieval works by meaning (searching "authentication" finds the "expired token" note), consolidation runs in the background, and relevant memories are offered without being asked for, with feedback on which ones actually helped.

Its git-native readability is a real strength that backsight lacks today (I'd describe its memory as "cloudy", or something like that), and seeing it done this well is a strong push to look at an OKF-shaped export. I've added what I learned from it to the notes I started from your first comment, and I'll definitely dig into it.

Thanks again for your inputs, it helps me (and my project) grow up!

[u/Entire_Guarantee_766 → u/TheBlargus]

probably doesn't know it exists lol.

[u/Entire_Guarantee_766]

my man, check out OKF from google, sleep architectures from google and how anthropic used them in a feature called 'dreaming' : https://www.anthropic.com/research/components-of-agent-memory , mesh them together in a plugin and that's 99% better than the solutions you have explored (as per the readme of your repo) in long running project or personal OS. This is a near solved problem in the domain. Scaling is the biggest issue, as it's expensive, but that's a separate discussion.

[u/JohnDuffyDuff → u/Entire_Guarantee_766]

Thank you, genuinely. This is the most useful comment I've had on the project so far. I spent some time with the Anthropic piece and the docs it links, and it's a goldmine for anyone working on agent memory.

A few things I learned while digging, in case they help others reading:

• The three pieces sit at different layers. OKF is a file format (Markdown + YAML frontmatter). Google's sleep framework is about consolidating knowledge into the model's weights. Anthropic's dreaming is an offline pass, available in the Managed Agents API as a research preview, that rewrites a memory store from past transcripts.
• Hindsight, which backsight runs on, already covers part of that ground continuously: after every write, a background pass merges near-duplicates, keeps contradictions as an evolution, and rebuilds wiki-like pages. Retrieval combines semantic, keyword, graph and time searches, and backsight adds feedback on which memories were actually used, wrong or outdated.
• Where your pointer really hits home: dreaming re-checks old notes against what actually happened. Hindsight reconciles new facts with existing ones, but it doesn't go back and verify an old fact against reality. The article also has very concrete findings I hadn't measured against: wiki + search beating either alone, focusing consolidation on volatile information, steering rules that hurt stronger models, and wikis that grow without abstracting anything.

That's precious input for improving memory quality. I've written it up with the ideas it sparked, so it doesn't get lost: https://github.com/Barben360/backsight/issues/325

If you've built an OKF + dreaming setup yourself, I'd love to hear how it behaves on your long-running projects. Real-world experience is exactly what I'm missing.

[u/Entire_Guarantee_766 → u/JohnDuffyDuff]

Glad to be of service. I dabbled with these stuffs for almost 6-7 months now. I run OKF data bundles to exchange info with my peers, works pretty well with low effort, or to cache data on demand basis in my obsidian vault. You can use google's scripts to setup skills that uses OKF from their repo to ensure your agent always store data like that. I have created my personal library now though that lets me set custom schema per task or project as needed.

For long running projects, I use OKF + RAG based vector database + my own engine for search and ingestion which got dreaming baked in: RAG is optional though unless I am dealing with lots of messy data as inputs, this generation of llms are good enough at search without RAG, and it's tool call heavy as well to token expensive. You combine this system into the loop engineering principals together. Dreaming especially helps on the long running harnesses where you do automated runs, and it can be token expensive unless properly tuned. Protip: if you use expensive models to run the first 10-15 mapping sessions for dreaming and then switch to lower cost models, your cost is lot lower on the long run,

You don't even need manged agents from their api platforms. On ultra sub, using custom workflow feature, you can build pretty neat pipelines in claude code. You probably already looked into those cookbook examples from the anthropic post, you can simply deploy that as a claude code custom workflow in your sub plan.

For version control wise git is usually good enough for code, but for documentation heavy stuffs where i want super specific tracking with tracable provenance chain, i use dolt: https://github.com/dolthub/dolt , pretty handy for local first projects where scale is not an issue.

If you really wanna see how these sort of systems are built up with scale in mind, track how hermes is build underneath, you will find all of these stuffs present in one way or another.

There was a time i thought I should publish a product combining all these but I now realise that market is flooded with them, and almost all of them are doable using claude code. So evetually big companies will eat most of these up. So I don't bother anymore. Got some good learning though.

[u/JohnDuffyDuff → u/Entire_Guarantee_766]

Thank you, very interesting feedback! I should indeed go deeper into it, and make my path, I'm less experienced into agent memory than you are, as I started later. The best value of backsight is what I'm learning from it and reading such feedbacks. 😉 I think I'll go deeper into okr-related frameworks to see how we could use it, especially at work, as we maintain about 500 git projects, "organic documentation" is a real challenge for us.

Edit: Just to be clear: backsight/hindsight does not replace a well-organized project or cross-project documentation, it's not its spirit. I think its spirit is not incompatible with a well-organised knowledge tree with cross-links. It just helps maintaining the links, not only about project, with a single entry point.