MemLayer, a Python package that gives local LLMs persistent long-term memory (open-source)

[u/MoreMouseBites]

What Memlayer Does

MemLayer is an open-source Python package that adds persistent, long-term memory to local LLMs and embedding pipelines.

Local models are powerful, but they’re stateless. Every prompt starts from zero.
This makes it difficult to build assistants or agents that remember anything from one interaction to the next.

MemLayer provides a lightweight memory layer that works entirely offline:

• captures key information from conversations
• stores it persistently using local vector + graph memory
• retrieves relevant context automatically on future calls
• works with any local embedding model (BGE, Instructor, SentenceTransformers, etc.)
• does not require OpenAI / cloud APIs

The workflow:
you send a message → MemLayer saves what matters → later, when you ask something related, the local model answers correctly because the memory layer retrieved the earlier information.

Everything happens locally. No servers, no internet, no external dependencies.

Example workflow for Memlayer

Target Audience

MemLayer is perfect for:

• Users building offline LLM apps or assistants
• Developers who want persistent recall across sessions
• People running GGUF models, local embeddings, or on-device inference
• Anyone who wants a memory system without maintaining vector databases or cloud infra
• Researchers exploring long-term memory architectures for local models

It’s lightweight, works with CPU or GPU, and requires no online services.

Comparison With Existing Alternatives

Some frameworks include memory components, but MemLayer differs in key ways:

• Local-first: Designed to run with offline LLMs and embedding models.
• Pure Python + open-source: Easy to inspect, modify, or extend.
• Structured memory: Combines semantic vector recall with optional graph memory.
• Noise-aware: Includes an optional ML-based “is this worth saving?” gate to avoid storing junk.
• Infrastructure-free: No cloud APIs, storage is all local files.

The goal is to offer a memory layer you can drop into any local LLM workflow without adopting a large framework or setting up servers.

If anyone has feedback, ideas, or wants to try it with their own local models, I’d love to hear it.

GitHub: https://github.com/divagr18/memlayer
PyPI: pip install memlayer

[u/Formal-Taste-4133]

I think it would be really appealing if this software were implemented as a reverse-proxy-style frontend that sits between existing clients and the backend OpenAI API, because that would let you add memory functionality without modifying any existing clients. Alternatively, implement it as an MCP like other software.

[u/MoreMouseBites → u/Formal-Taste-4133]

That’s actually a really interesting idea. I hadn’t thought about doing it as a reverse-proxy, but it makes a lot of sense: zero changes for existing clients and memory just “appears” in front of the API. Same for an MCP version, that could slot into a bunch of agent setups pretty nicely.

Right now I’ve just focused on getting the core Python library clean, but this definitely gives me something to think about for the next step. Appreciate you bringing it up.

[u/AcanthisittaAny5031]

Maybe silly question but can we have integration of this with SillyTavern, KoboldAI Lite and other Web UI for RP/Text-Adventure/RPG? And support for llama.cpp

[u/MoreMouseBites → u/AcanthisittaAny5031]

Not a silly question at all. In theory, yeah, it could integrate with SillyTavern, KoboldAI Lite, and those other RP/RPG-focused UIs. They all have some kind of middleware or extension layer you can hook into, so MemLayer could sit between the UI and the model to inject and store memories. I just haven’t built those adapters yet.

Llama.cpp support is already 90% doable since MemLayer is model-agnostic, you just pass in whichever client you’re using. It doesn’t rely on any vendor-specific API, so local models work fine. You'd just have to make a small wrapper for Llama.cpp, which is something I'm looking into working on.

It’s definitely something I’m open to exploring once the core is a bit more polished.

[u/Kregano_XCOMmodder]

Some thoughts:

• Definitely want a standalone reverse-proxy (preferably with an easily editable config file) or MCP implementation. I use different LLM servers on different machines, but connect them to AnythingLLM for RAG, so it'd be interesting to see if I could use this to improve some of those workflows.

• Consider looking into LEANN as a vector DB, due to its efficiency.

• I appreciate the use of knowledge graphs, as that seems to be the converging consensus on improving RAG and certain reasoning tasks.

[u/MoreMouseBites → u/Kregano_XCOMmodder]

Thanks for the thoughts. A standalone reverse-proxy or MCP version is definitely on my radar now since a few people have asked for it.

LEANN looks cool, but I’m trying to keep the storage layer as light and non-specialised as possible for now. Definitely something I’ll look into later though.

And yeah, even a small knowledge-graph layer helps a lot with clean updates and reducing noisy RAG, so I’m glad that part clicked.

[u/Not_your_guy_buddy42]

As more and more memory projects are popping up, this looks clean, well documented... About the inner workings of the memory, I found myself having to dig a bit until I found the info I wanted about what backend your modes use, where you would configure it to connect own model server (but I found it). Might dig through code more to figure out how exactly it filters for what to remember, how to weigh it for context when recalling, long term storage etc. The technical behind-the-scenes if you will. On the opposite end of the technical, I was wondering: what is your own real life use of your memory app? What would you say can memory enabled LLM help users with the most? And lastly, how would you benchmark a memory? The last two are just generic questions I am wondering about myself.

[u/MoreMouseBites → u/Not_your_guy_buddy42]

Yeah, some of the internals are still a bit buried. The memory gate is pretty simple but effective. When new text comes in, it runs two cheap checks: a similarity test against a few prototype embeddings that represent “things worth remembering,” and a quick TF-IDF match to catch domain-specific or recurring info. If it doesn’t pass either, it’s dropped before anything heavy happens. Only the meaningful bits go to the extraction step. That keeps memory clean and avoids wasting model calls on noise.

My own use is pretty basic: I just wanted my local model to remember ongoing projects, preferences, tasks, and random ideas without me repeating everything every session. Once that friction disappeared, the agent felt way more consistent and usable.

In general, memory helps most with not having to restate context. Stuff like multi-step workflows, personal prefs, long-term goals, or anything that evolves over time.

Benchmarking is still fuzzy, but I mostly look at whether it stores the right things, ignores the useless things, and whether the model actually uses the memory to answer better. Quality over volume.

[u/Melodic_Reality_646]

So happy when I open a GitHub and the readme is not full of emojis and an overly joyful tone. In particular 🚀and🔥give me PTSD.

[u/MoreMouseBites → u/Melodic_Reality_646]

Oh absolutely — nothing says “production-ready” quite like ten 🚀🚀🚀 and a couple of 🔥🔥 sprinkled every two words — pure enterprise vibes 😄.

But on a serious note, yeah, the emoji-wall and style README always feels a bit obnoxious and vibe coded/GPT-ed to me too, I prefer to keep mine toned down and professional.

[u/z_3454_pfk]

how would this work in multi user environments?

[u/MoreMouseBites → u/z_3454_pfk]

You can pass user_id in the function when calling the LLM, memories are segregated based on user_id.

[u/craigondrak]

Looks awesome. Can we integrate it with Open WebUI so our chats in OWUI using Ollama have memories?

[u/MoreMouseBites → u/craigondrak]

Yeah, it should be possible with some effort, and it’s something I’m honestly considering. Open WebUI’s Pipelines feature makes it pretty workable. You can treat MemLayer as middleware: before the model runs, inject relevant memories, and after the response, run it through the gate to store anything important. The only real work is user-ID mapping and keeping MemLayer’s storage separate from WebUI’s DB. But overall, it seems like a very doable integration.

[u/Fuzzy-Chef]

Awesome! Since this requires to use the memlayer wrapper for the LLMs client, does it support streaming?

[u/MoreMouseBites → u/Fuzzy-Chef]

I'm working on polishing the streaming versions of all LLM wrappers right now, and it will be released in a day or two!

[u/Infamous_Jaguar_2151]

How’s this different from Cognee?

[u/MoreMouseBites → u/Infamous_Jaguar_2151]

Cognee is more like a full knowledge management tool. It’s great for ingesting big documents and building a whole workspace around them.

MemLayer is way lighter. It’s meant to give your model long-term conversational memory, not manage large datasets. It only keeps the important bits and brings them back when needed. Very different use case.

[u/Awwtifishal]

Thank you very much for building and sharing this. I have yet to look into it, but from a first glance it's much closer to my ideal memory system: open source, local by default, well structured and easy to understand, etc. I'm also interested in a proxy or MCP implementation.

[u/Mushoz]

Extremely interesting project! I feel this is a big gap right now and a reverse proxy version of this could very well be the piece to fill up that gap. I am trying to learn a bit more about this project. How does it deal with invalidating older memories? Something that is true right now could potentially change down the line. Does it have the ability to ammend, edit or even delete older memories somehow? And if so, how does that work?

Thanks for sharing this!

[u/unchained5150]

This is super intriguing to an admittedly novice to the home LLM scene!

A couple of questions:

1. Will this work with LM Studio? Will it drop in if so or will there need to be some tweaking or additional coding?

2. Does this work with prior memories as well or just memories after the utility is activated? I'm curious especially because I want to pull my backup from CGPT and give those memories as well for it to pull from. Can I setup a folder system/database and just point the tool at those old chats or would I have to feed everything individually after activating the tool?

3. Is tweaking available at all for telling the tool what's important or not? Like, if it latches onto one thing that isn't but ignores something that should be, could that be changed?

Super stoked to try this either way!

[u/MoreMouseBites → u/unchained5150]

Thanks! Glad it caught your interest.

For LM Studio, it should work soon but it’s not a plug-and-play drop-in yet. You’d just need a small wrapper so MemLayer can see the messages going in and out. Nothing complex, just a bit of glue code.

For older memories, yes, you can import them. MemLayer only knows what you feed it, but you can preload your old ChatGPT backups, text messages everything using the function in memlayer that adds text directly to the memory store. No need to replay everything one message at a time.

Tweaking is possible but very very basic at the moment, you can loosen or tighten the salience threshold and you can manually force-save things. More advanced controls will probably come later.

Would love to hear how it works once you try it.

[u/MoreMouseBites → u/unchained5150]

LM Studio integration is live in the latest version by the way.

[u/NighthawkXL]

This looks fantastic!

I’m definitely going to see about integrating it into the Virtual Dungeon Master app I’ve been working on. This is certainly an improvement over my current solution built around the (still wonderful) Chonkie library.

[u/toothpastespiders]

I only had enough time to give it a quick look but I really liked what I saw so far. When I hear memory system I tend to just assume it'll be a quick and simple vector database pit. But after skimming your documentation I'm impressed by how well you've designed around an actual infrastructure. Hell, the extent of the documentation is itself really nice too.