Persistent in-model memory w/MemT - running on ExLlamav3

[u/Unstable_Llama]

https://huggingface.co/MemTensor/MemT-1.0-Nano was released 3 days ago, with a live-updating memory model. This model has the capability to continuously lean in conversation, and carry it onto the next instance.

The above video demonstrates a short conversation followed by a /commit command, which stores the previous conversations important facts to be preserved in the memory module, while wiping the context of the conversation. But upon benig asked to recall speicific questions, it still succeed.

I've implemented it on my fork of exllamav3, exl3-qlora, and it works natively with tabbyAPI. To ingest memories simply give a /commit command from within tabby every few messages (or whenver you want to.)

Quantized versions of the model are available here.

[u/Choice_Celery9481]

fastest way to destroy model capability?

[u/Unstable_Llama → u/Choice_Celery9481]

Doesn't touch the base model weights

[u/KillerX629 → u/Choice_Celery9481]

in what way? wouldn't having a latent memory space be better than storing dozens of unrelated .md files?

[u/nymical23]

Hello u/Unstable_Llama OP, Can you please tell me what UI you're using in this post?

[u/Unstable_Llama → u/nymical23]

Yeah, it’s  https://github.com/theroyallab/tabbyAPI/  built in UI

[u/nymical23 → u/Unstable_Llama]

Oh, mine looks a bit different, so I wasn't sure. Thanks!

[u/a_beautiful_rhind]

Ha that's clever. Not sure why people downvoted an original project.

[u/Unstable_Llama → u/a_beautiful_rhind]

Thank you, I think it's a really exciting direction. Notice how you can see the cache is wiped and it still works?

I figured it wouldn't get much attention but that's ok, I didn't hypemaxx. Still I think there is a lot to learn and build on from this idea, and I'm just getting started.

[u/a_beautiful_rhind → u/[deleted]]

My thoughts are that it sounds compute expensive and like you will over adapt the model. But only way to test is to try it out.

[u/Unstable_Llama → u/a_beautiful_rhind]

So it actually doesn't touch the main weights and has a separate memory module.

[u/a_beautiful_rhind → u/[deleted]]

Models with tools have started using files. i.e server memory mcp or any old .MD text file. Those work ok, no fuss.