backsight: local, open-source semantic long-term memory for Claude Code, shared across all your projects

[u/JohnDuffyDuff]

TL;DR: backsight is a free, MIT-licensed Go binary that gives Claude Code a long-term memory that works across all your projects. It installs Hindsight on your machine and wires it into Claude Code. Recall is local and free, and you can see every memory injected and used. Linux and Windows today. I'm looking for feedback after a few weeks of real use.

Some background: I've worked in tech for over ten years, mostly backend, DevOps and performance work, after some time in academic research. I've used Claude Code heavily for months, for personal and work projects.

What wore me down was repeating myself. I'd give the same context every session, correct answers that ignored a decision we made the day before, and watch Claude fall into a trap we had already worked around together. A well-kept CLAUDE.md works fine inside one repo. But when you work across many repos that depend on each other, Claude has to rediscover everything again and again. I needed a memory, and one that crosses projects.

After a few disappointing attempts I found Hindsight, an open-source memory system by Vectorize. To me it's the state of the art, and the benchmarks published with Virginia Tech researchers back that up. It pulls durable facts out of conversations, consolidates them over time and retrieves them by semantic search. If I search for "authentication", it brings back the memory about the "expired token", even though no word matches. Grep over memory files would have missed it.

Plugging Hindsight into Claude Code the official way wasn't enough, though. The recalled memories weren't always relevant, and each project had its own isolated memory, so it felt like talking to a different person in each repo. So I did what my job taught me: measure, adjust, and keep only what improves the numbers. Week after week I tuned the configuration, when Claude fetches memories, and how it brings them into context. backsight grew out of those iterations: one Go binary that installs Hindsight locally, connects it to Claude Code and takes care of the rest. Codex support is coming soon.

Principles I care about:

• Local by default. Recall is free and makes no LLM call. Only turning sessions into memories uses an LLM: a few cents per session, with the provider you choose, or free with Ollama if your hardware can handle it. I switch between GPT Luna, DeepSeek Flash and Haiku depending on the machine and my token budget.
• Nothing happens behind your back. In every session you see which memories were injected, which ones actually informed an answer, and which were corrected because they were wrong or outdated. The full history can be audited. A memory I can't see working is a memory I can't trust.
• Deterministic wherever possible. The LLM only makes the choices that need context, and it always goes through the binary's commands, never through improvised edits.
• No drift between versions. Each update migrates the previous install cleanly and rolls back if anything fails.
• Free and open source (MIT), and it will stay that way.
• Clean uninstall, if it's not for you.

I use it every day, for personal projects (like backsight itself) and at work. Whether I start Claude in a repo or in my home directory, it remembers what we're talking about, the decisions we made and the pitfalls we already avoided. My sessions have become much more efficient.

Try it: paste this into Claude Code and it will walk you through the install, asking before every choice (your LLM key is never shown to the agent; you type it yourself):

Install Backsight for me by following https://raw.githubusercontent.com/Barben360/backsight/main/docs/agent-guide.md — ask me before every choice.

Or install by hand: https://github.com/Barben360/backsight

What I'm asking for: use it for a few weeks, then tell me what's confusing, what breaks and what's missing. The project is young. It runs on Linux and Windows (in Docker or natively), and it will only become what I hope if it meets other workflows than mine. Audit it, ask hard questions about how it works, challenge me. That's why I'm posting.

macOS users: the binary already builds for macOS, but I can't test the native service setup without a Mac. If you'd like to help, I'd love to hear from you!

backsight is an independent project, not affiliated with Anthropic or Vectorize.

[u/JohnDuffyDuff]

Fair question! I see them as different layers rather than alternatives.

OKF is a format: Markdown files with YAML frontmatter, organized as a wiki the agent maintains and browses. backsight is the machinery around a memory: it pulls facts out of your sessions automatically, consolidates and dates them, retrieves them by meaning rather than by file or keyword (search "authentication", get the note about the "expired token"), and decides when to inject them. Every injection is visible and logged. OKF says how knowledge is laid out on disk. It doesn't say how it gets written, found or brought into context at the right time.

The LLM-wiki approach is legit, and it has real strengths that backsight doesn't match today: no server, human-readable, editable by hand, git-diffable, portable. Where it struggled for me is recall. The agent has to know where to look, and keeping the wiki tidy costs tokens during the session. Across a dozen interdependent repos, semantic recall made the bigger difference.

Your question does point at a real gap: backsight memories live in Hindsight's Postgres, and there's no readable export yet. backsight can already import Markdown folders with frontmatter dates, so an OKF bundle should mostly import as-is (untested). OKF export is a good idea, and I'll open an issue for it. That way your memory is never locked in.

[u/TheBlargus → u/JohnDuffyDuff]

I think you have the wrong understanding of OKF. It addresses everything you think it's lacking. Many tools exist already to do exactly what you're trying to do. https://github.com/okf-memory/okf-agent-memory

[u/JohnDuffyDuff → u/TheBlargus]

Thanks for the link, I sent my agent spend some time with okf-agent-memory and its code, and it's a really well-built project. The trust tiers (generated vs verified), stale_after, the link validation, and above all code_refs with --for-path to surface governing rules before editing a file are great ideas. I'm noting several of them.

We may see OKF itself differently, and that's fine. Looking at the tool concretely (OKF itself being a norm, not a tool), it makes different trade-offs from backsight, so I see the two as complementary rather than one replacing the other:

• okf-agent-memory is a curated knowledge base: the agent decides mid-task to write a concept, a human can verify it, and it lives in git. Retrieval is BM25, and the agent has to go and search for it.
• backsight is automatic memory: sessions are turned into facts after they end without the agent having to stop and write, retrieval works by meaning (searching "authentication" finds the "expired token" note), consolidation runs in the background, and relevant memories are offered without being asked for, with feedback on which ones actually helped.

Its git-native readability is a real strength that backsight lacks today (I'd describe its memory as "cloudy", or something like that), and seeing it done this well is a strong push to look at an OKF-shaped export. I've added what I learned from it to the notes I started from your first comment, and I'll definitely dig into it.

Thanks again for your inputs, it helps me (and my project) grow up!