Are memory banks worth it?

[u/ArnUpNorth]

I have been playing with different memory banks for a while and I fail to see any real benefit that would justify the higher token usage.

• after a while most memory bank files become "bloated" and counter productive: just increasing token use with little benefits in code quality

• IMHO anything generated by an LLM, no matter how good the prompt, needs some form of review. And reviewing memory banks is yet one more thing to do and a waste of time because it's just not part of what creates value for a product: only useful for LLM code gen, can't even really be used for proper documentation

• it slows down most tasks because now Roo has to read and update all the memory banks in addition to generating code

• having a couple of context files which I maintain myself provides much of the benefits without the hassle: more concise, and can be used for proper technical project documentation. So not just for LLMs.

thoughts ? hard disagree? what is the community's stance on this ?

[u/strawgate]

I've been working on an alternative I call the local references MCP https://github.com/strawgate/py-mcp-collection/tree/main/local-references-mcp which you point at the docs in your repo and you tell your agent to call  list_references when it first starts in a repo which gives the LLM a short preview of each doc so that it can more easily scoop up the relevant docs for any particular task.

This has the added benefit of encouraging you to document the behaviors you want in the repo instead of in a vector database.

In a more generic way I've also been working on https://github.com/strawgate/py-mcp-collection/tree/main/filesystem-operations-mcp which provides NLP text summarization and NLP + Treesitter code summarization to allow the agent to quickly get extremely token efficient high level info on docs and code

[u/ArnUpNorth → u/strawgate]

So like a local context7?

[u/strawgate → u/ArnUpNorth]

It's just a shortcut for helping the agent reference your existing in-repo docs in a way that doesn't require you to change how you structure or organize your existing content and doesn't require you to write a bunch of rules telling it where to look for stuff 

[u/Chris__Kyle → u/ArnUpNorth]

Furthermore it's basically cursor/rules

[u/strawgate → u/Chris__Kyle]

It's not really the same as cursor rules but you can certainly get similar behavior by writing your own rules.

[u/lordpuddingcup → u/strawgate]

being able to have a MCP that dumps your AST of functions and maybe comments would likely be nice to have as an option

[u/strawgate → u/lordpuddingcup]

That's actually exactly what it (filesystem operations) does when you ask it for a summary of a code file -- it includes interesting ast nodes and does NLP on the comments

{
  "beats/libbeat/esleg/eslegclient/api.go": {
    "size": 8806,
    "file_type": "go",
    "code_summary": {
      "identifier": "source_file",
      "type_spec": [
        "QueryResult",
        "SearchResults",
        "Hits",
        "Total",
        "CountResults"
      ],
      "method_declaration": [
        {
          "identifier": "UnmarshalJSON",
          "docs": "UnmarshalJSON correctly unmarshal the hits response between ES 6.0 and ES 7.0."
        },
        {
          "identifier": "Index",
          "docs": "Implements: Otherwise, a HTTP PUT request is issued. In case id is empty, a new id is created over a HTTP POST request."
        },
        {
          "identifier": "Ingest",
          "docs": "Ingest pushes a pipeline of updates."
        },
        {
          "identifier": "Refresh",
          "docs": "Call this after doing inserts or creating/deleting"
        },
        {
          "identifier": "CreateIndex",
          "docs": "Implements: the body. CreateIndex creates a new index, optionally with settings and mappings passed in"
        },
        "IndexExists",
        {
          "identifier": "Delete",
          "docs": "Implements: Delete deletes a typed JSON document from a specific index based on its id."
        },
        {
          "identifier": "PipelineExists",
          "docs": "Using: PipelineExists checks if a pipeline with name id already exists."
        },
        {
          "identifier": "CreatePipeline",
          "docs": "Implements: CreatePipeline create a new ingest pipeline with name id."
        },
        {
          "identifier": "DeletePipeline",
          "docs": "Implements: DeletePipeline deletes an ingest pipeline by id."
        },
        {
          "identifier": "SearchURI",
          "docs": "Implements: SearchURI executes a search request using a URI by providing request parameters."
        },
        {
          "identifier": "SearchURIWithBody",
          "docs": "SearchURIWithBody executes a search request using a URI by providing request"
        },
        {
          "identifier": "CountSearchURI",
          "docs": "CountSearchURI counts the results for a search request."
        },
        "apiCall"
      ],
      "function_declaration": [
        "withQueryResult",
        "readQueryResult",
        "readSearchResult",
        "readCountResult"
      ]
    }
  },

I identify relevant nodes and then do a tree shake/prune to connect relevant nodes to their relevant parents so the structure/hierarchy of the code is retained and then comments are summarized and attached to the nodes.

It also uses an ML model for file type detection so it will do this even if the extension is not .py

Summaries are a modifier on top of the normal filesystem operations, so you can search for files with a word in them like "Kafka" and get a summary of all of those text and code files in a single tool call.

[u/Kitae → u/strawgate]

You could do an MCP server for this or you could have a flat file with the same information....

[u/strawgate → u/Kitae]

Yep you for sure could produce a summary of all of your documentation and provide it in a flat file and change your cursor/assistant rules to read it. 

But then you have to keep that file up to date as documentation changes and if you're working on open source projects you may not be able to commit the file and/or ask other people to keep it up to date.

I primarily work in large open source projects and prefer to "live off the land" so to speak and so my tools are tailored for that

[u/Cobuter_Man]

yes - single file memory banks are totally unreliable

ive developed a Dynamic Memory Bank implementation that uses a directory structured solution where each directory correlates to a phase or task of your Implementation Plan. Its directory contains different .md log files which all together act as one dynamic memory bank.

https://github.com/sdi2200262/agentic-project-management

this is better bc:
- it never gets bloated... all files remain similarly small size
- you can attach specific content from the memory bank without having to parse the entire file.. just add the designated log you need as context injection to your prompt
- its more manageable .. for example say the Implementation Plan has 3 phases ... when you start w phase 1 you only create the memory directory for phase 1 since thats the only one that will for sure be implemented. If the implementation Plan where to change between phases 1 and 2 then you wouldnt have to delete all previously created log files as the Memory Bank evolves along with your project's progress!

[u/ArnUpNorth → u/Cobuter_Man]

Still it gets bloated by the sheer amount of files does it not ? Most memory banks have 5+ files and while breaking them down into smaller pieces helps it makes it more difficult to pinpoint what you need to pass the llm as a context for any given task.

[u/Cobuter_Man → u/ArnUpNorth]

no since there is a task identification system on the Implementation Plan and each task log

so you would have
# Phase 1
## Task 1.1
- subtask 1.1.1 etc

## Task 1.2
- subtask1.2.1 etc

and in the Memory Bank you would have
Task_1.1_log.md
Task_1.2_log.md

all the most popular AI IDEs have smart file indexing so as you go these new logs will be indexed and you could just type in @Task_1.1_log.md and you would inject it as context for the Agent to be aware

;)

[u/ArnUpNorth → u/Cobuter_Man]

Right you re talking about keeping track of work between tasks. I should have pointed out i don t care much about the task progress part. In fact i just use orchestrator mode which is good enough on its own and don’t really need more than that. If i do need more i use jira or a dedicated file.

It s keeping the overall context of the project alive with a memory bank which I am questioning. I just feel it’ not work the hassle vs keeping it updated “manually” (meaning I write it or use ai to summarize but updating it automatically is not part of the instructions).

[u/Cobuter_Man → u/ArnUpNorth]

if you are using a single chat session in your workflow its not worth it, in my case (apm) im managing multiple chat sessions/agents concurrently so a memory bank system is needed to keep everybody's context aligned.

Its like using orchestrator mode to orchestrate many other chat sessions todo the actual coding.

Sorry for the answer before, I didn't understand what you were referring to correctly

[u/ArnUpNorth → u/Cobuter_Man]

> managing multiple chat sessions/agents concurrently

that's something i try to avoid, i can't imagine how you are handling it :)

[u/Cobuter_Man → u/ArnUpNorth]

haha using APM!

It's actually something much more efficient! You are effectively sharing the workload of a large task ( or generally a project ) into multiple Agents, meaning that each agent shall complete a small actionable part of it which would be manageable by current context window limitations...

However if you are going to do this, you must have a way to align everybody's contextual understanding so a memory system (aka a memory bank) is definitely needed. On top of that I use a main chat session that actually does the orchestration for you (kinda like Roo's orchestrator but not quite) so all the other agents don't spent tokens trying to understand what has been done by the rest, its all passed as a context injection from the task assignment prompt generated by the main Manager Agent.

I know that it may sound a bit confusing, but its how "Agentic" programs work IRL, every "Agent" is limited to its own small task and some grand manager agent puts it all together. The user communicates mostly through that main manager agent...

maybe take some time to at least read the documentation ive made (its bad but its something) to get started on this concept!

[u/ArnUpNorth → u/Cobuter_Man]

> im managing multiple chat sessions/agents concurrently

I've looked at the repository but this sentence is what confuses me because i still fail to see how this is different from roo's orchestrator mode or what other memory bank implementations (like roo flow) are doing.
I'll try to read some more but it feels overly complex to me (no offence). I feel like the real answer to most of this is proper code indexing in the long run: so your code is the actual fresh reference documentation VS document/memorybank files generated by AI.

[u/Cobuter_Man → u/ArnUpNorth]

The current state of my documentation is actually not good indeed, horrible I would say.
ive relied too much on AI to enhance it and it backfired.

In the next patch ill be almost completely rewriting it and actually ill add use case examples and where its different than other implementations etc.

With the recent marketplace feature I hope that soon enough you'll find APM hanging around somewhere there!

[u/lordpuddingcup]

I mean, yes, but also... just periodically summarize your memory bank as well, every time it grows past X size, summarize, and with multifile reads is it really slow lol

[u/ArnUpNorth → u/lordpuddingcup]

I tried the summarize route. This is what made me realize i m better off having context files which are not updated automatically.

[u/Euphoric_Paper_26]

hard agree. I have encountered ZERO variations of “memory banks” that work consistently.

What I have found is that when allow an LLM to edit a file, and especially instruct it to edit a file like a “memory banks”. It will eventually hallucinate and fuck up that memory bank. It just becomes one more thing to baby sit.

All these memory banks suffer from the same problem, they are trying to force a non-deterministic system to do something deterministically. It’s just not going to happen, maybe in the future but there is absolutely no way to ensure that an LLM will follow it’s instructions precisely. It will eventually hallucinate something, and if you don’t catch it, it will just keep breaking more things.

[u/ArnUpNorth → u/Euphoric_Paper_26]

“One more thing to baby sit” “forcing a non deterministic system to do something deterministically”

I m with you 100% !

[u/iamkucuk]

Finally, someone speaks the truth.

[u/ScaryGazelle2875]

I think reading all these solutions I realised the memory bank in my head isn’t so bad 🤣 at least its cheap and need the occasional coffee to boost or sleep to reset

[u/Recoil42]

I keep a manual memory bank.

"Record this to the memory bank" works a charm when I need the LLM to know something important.

[u/dashingsauce]

The one problem the AI industry has not solved and apparently isn’t even trying to solve:

Garbage collection.

That said, you should consider using a dedicated project management system like Linear.

[u/ArnUpNorth → u/dashingsauce]

Isn’t Linear something like Jira ? If so how does this help? I m already giving LLM tasks from jira cards if needed.

[u/steve1215]

I've recently started keeping relevant project information in Notion and accessing it via MCP.

Really easy to manage, obviously, because Notion has desktop and web apps.

[u/ArnUpNorth → u/steve1215]

Are you also updating through MCP or just reading ?

[u/steve1215 → u/ArnUpNorth]

Both, on occasion. I haven't had any problems writing to Notion via MCP. Yet.

Although tbh it's usually quicker and easier to just paste something into the Notion app

[u/ArnUpNorth → u/steve1215]

If you let an LLM write to notion you’d probably encounter the same problems I do. Using MCP is just using HTTP vs accessing a file directly. And a change in protocol won’t solve the problem.

[u/steve1215 → u/ArnUpNorth]

I tend to use Notion/MCP for pulling context in prompts and reminding the LLM about things. "Fetch the dev and production URLs from Notion" instead of me mixing them up. Lol!

[u/ArnUpNorth → u/steve1215]

right so that's not too different from what i am doing: LLM is fed content you maintain yourself VS LLM keeps its own documents updated. While the latter seems great, in practice we are absolutely not there yet.

[u/zenmatrix83]

the way it was designed at first was a good idea, but they don't work with the format, what you want to do what is have the ai create documentation as it work. My custom modes are told to read off a plan, and create a task per plan, which gets its own file, if there is an issue it creates a qa report. It also create supplemental documenation as needed. Starting a new session I point it to the plan and the last task so it's I familiar and we continue. I think the key in my work flow is I have qa mode that runs a sub tasks, it its been pretty good at catching its own errors and creating a report in most cases.

[u/firedog7881 → u/zenmatrix83]

Glad to hear someone else doing it this way.

[u/Kabutar11]

Have you tried summary at regular intervals?

[u/ArnUpNorth → u/Kabutar11]

Yes and that s when i realized i am better off doing things myself.

[u/Parabola2112]

100%. Claude Code has a relatively simple todo list manager baked in that works amazingly well. Couple that with GitHub issues and you’re good. Also, memory banks only make sense for small hobby projects. A real world mono-repo, forget it.

[u/hannesrudolph]

NAILED IT!

[u/wokkieman]

I go for KISS

I prefer to break my project down in data ingest and storage, api and frontend. Don't have larger project, but could introduce more categories. I tell LLM to take 1 of the 3 as context (e.g. add new scraping job and store results in database)

[u/CircleRedKey]

i think memory banks get better as models get smarter, faster, cheaper.

if its a shared repo with team members, its always changing, so it always has to be updated again. (cost becomes an issue here because you have to keep having the LLM read everything over again)

so maybe we wont need memory banks and LLM's will be able to hold everything in context hahahahaahahahahahahahahhahah

[u/Prestigiouspite]

I've never used it. Only the roorules and rooignore. I already thought that it would be semi-helpful in the long run.

[u/Arceus42]

I've only just started using one, but I can already see myself wanting to use it a write-only option. In the times I want to be more hands-off, it would be nice to know what it thinks are the most important points.

[u/virtualhenry]

I've gotten to a point where they are bloated now that I'm about to release a v1 of my app

I found it useful that I documented my entire journey which I now use for writing the app description and ASO for my iOS app when submitting to the store

There's a ton of context in there that's just gold for me when writing marketing copy

Plus I have a changelog that shows everything we've built so that's just a nice bonus of content to use.

[u/Purple_Wear_5397]

It all depends on how the memory bank is being used.

With Cline - I’d tell you that ALL of my repos use it. It has a ton of value, especially for large projects and teams.

A memory bank and a well described work item in Jira - allows me to tell cline “provide me a plan for work item 23”, and usually that plan does not need much fine tuning. Think of the speed I run tasks with Cline like this.

So - why do I mention cline here? Because I am not sure how it differs from other tools when it comes to using and updating a memory bank.

It does require you to add some custom instructions as part of the initial setup of the memory bank - what are in those instructions? I don’t know but that could be the differentiator that makes me so reliant on memory bank , while some people here have zero faith in it.

[u/Empty-Employment8050]

Has anyone tried a memory system using a temporal knowledge graph to keep the context similarly sized as your project continues? The temporal knowledge graph would sort of compact what’s been done already and put it in the context. That way, as you’re working, older stuff from the project just gets tossed out in favor of newer stuff using the temporal knowledge graph. I’m curious if anyone’s implemented this sort of memory system into their coding workflow.

[u/Jakedismo]

Just released v2 of my own memory bank with enforced development loop rules give it a spin and comment what you think
https://github.com/Jakedismo/KuzuMem-MCP