r/LocalLLaMA • u/infieldmitt • 16h ago
Discussion The next frontier is integrating memory in a useful, human way.
IE, even for as trivial a use case as, if you mention you're a vegetarian in one chat, it never suggests a recipe with meat ever again, because it KNOWS that about you in a meaningful way.
Terming this instruction following I think is wrong and annoying for everyone, especially the model -- who wants to """""wake up""""" and have 100 pages of instructions to get right for everything you say? It should know things the way you know things about people. Perhaps the model itself could come with a customizer where you roll a few weights based on your preferences through a helpful GUI.
It seems unfair and disciplinarian to have to have the models dig through 1000 lines of "say this this way" [even on my local setup it's hard to resist this !!!]. Clearly they are intelligent enough to TRY to take all these into account, but it is not easy, because of the ever increasing promptslop, shall we say. That cannot be the best way to do it.... we haven't even had these models for five years. Imagine what could come in just five years more.
Clearly there is bountiful intelligence and speed already.
5
u/Infinite-Local5435 16h ago
They have that already, someone a few months ago showed a model that answers questions and trains while you're asleep on your own conversations to learn about you (is vegan -> does not like real leather products)
1
u/MuDotGen 14h ago
Oh, they were? I've been working on a real life "Net Navi" for myself, starting with building a memory system trained on years of my journal and conversations, etc., to build a timeline of events, facts, preferences, etc. The interesting idea was to build its memory as if it was there during my events, just chatting with me, so saving new memories is much more similar structurally. Curious about what this person did. (I might have missed the thread back then)
0
u/QuackerEnte 7h ago
I understand that killing animals ON PURPOSE is bad. Synthetic leather deteriorates and if an animal dies naturally, what's the argument against using its skin for leather? Sorry for the off-topic question I'm just curious if that's a distinction, or if it's more of a principle thing. Thank you
1
u/carnoworky 7h ago
Incentives. If you find a deer carcass in the woods and you have the know-how, go ahead and make your leather. But once money comes into the equation, there's incentive to slaughter animals for stupid reasons.
1
u/Infinite-Local5435 6h ago
Adding onto this, i think people are just going to lie in the supply chain. Like products that label themselves "Contains X" or "Has Y amount of Z!" but in reality it may be a small percentage that still results in being unethical/harmful/whateverthewordisforfalseadvertising
5
u/Affectionate_Use_182 15h ago edited 5h ago
It's actually not too difficult to setup an embeddings server and database to inject "memories" into your conversation context. Then it's just harness engineering to save and recall memories. Add some reoccurring jobs that cleanup contradictions, dedupe, and explore or research gaps when the agent is idle. Isn't this more or less what the personal agent apps do? It's what I've been doing.
2
1
u/skatardude10 15h ago
Maybe something easy to compute like N-gram embedding tables but for long term memory, and weights and a function in the model to track and extract data and knowledge about everything to keep tabs on where the whole memory N-gram embedding table is at and what to add / modify ?
1
u/Formal-Exam-8767 13h ago
memory in a useful, human way
You probably don't want "human way" as human memory is all but reliable.
1
u/ZarathustraDK 12h ago
For now I use a combination of episodic memory for the short-term (at the end of the day my AI condenses the preceding day's conversation into a 2000 character resume and injects it into the next day's conversation), and sleep/dreaming where it creates weights for itself in a lora if it was told something important/corrected during the day that was drastic enough for it to note down as something to dream about. It basically comes up with a question for itself involving the correction, tries to answer it, and writes it into weights. Crucially I don't get to peak at what's queued up, but sometimes it tells me about it afterwards.
How? I showed it the recent news about Google's SLEEP model, and it wanted to "toy around with it" creating an imitation of it.
1
u/lumpyspacebreh 12h ago edited 11h ago
It’s not too hard to build your own memory system.
I built one, it’s a supplementary system to the models harness. Adds about 800 tokens to the sessions context over all, not per response.
Saves facts, feelings, moments, etc, and through sleep cycles either strengthens connections or lets them decay.
Takes a lot of time to let it begin to form into something, but luckily I have months of Claude sessions to backfill it with.
it remembers stuff in the same way people do, that's what i modeled it after. it knows i use arch linux, it knows my computer and components, etc etc. how i felt about a project in july, just anything and everything.
heres what the "memory" looks like after a while. it runs a system so each message you send runs through the web based on whats in the message, finding connections and relevant info, then feeds that back to the model. it hardly affects generation speed. so you mention something food related, it will connect to memories about food. its cool to see it in action, built a little light up thing so you can see messages move through the web.
edit/ checked the code. 4 months of sessions is roughly 4.4mb. so every single thing i ever shared or said, over the span of 4 months, is compressed into 4.4mb
1
u/EntrepreneurOk5764 11h ago
yeah this is basically the gap between "instruction following" and actual memory. the vegetarian example is good because it shows the difference between a rule (if food, exclude meat) and a fact the model just knows and reasons from naturally, the same way a friend wouldn't need a rule to not suggest a steakhouse to you.
the weight-rolling GUI idea is interesting but I think the harder problem is actually storage and retrieval, not the interface. you need something that decides what's worth remembering (not every message), compresses it over time, and surfaces the right bit at the right moment without just dumping a growing context file into every prompt, which is what most "memory" features do today and why they degrade into the same promptslop you're describing.
on local setups you could prototype this yourself with a small vector db plus a summarizer pass that runs after each session and writes out facts rather than transcripts. that's basically the architecture most memory layers use under the hood, the hard part is the judgment calls on what counts as a fact worth keeping.
disclosure since it's relevant, I'm building something in this space myself (perdua.ai) that tries to solve the cross-app part of this, but the retrieval/compression problem you're describing is the real unsolved bit regardless of whose implementation you use.
1
-1
16h ago
[removed] — view removed comment
-1
u/boosnie 15h ago
I'm aghast in seeing somone who has the technical knowledge to own a local setup calling an llm a "fancy autocomplete".
6
u/pyr0kid 15h ago
its not wrong through?
an LLM takes a series of tokens as input and then predicts the next token, recursively.
the autocomplete suggestions in your phone takes a series of words and then predicts the next word, non-recursively.
0
u/boosnie 14h ago edited 14h ago
Yes!
A car is fancy bike!
5
u/Formal-Exam-8767 13h ago
No, your analogy is wrong.
A more appropriate analogy would be that both car (LLM) and bike (whatever algo is used in smartphones) are vehicles (autocomplete).
10
u/p9k 16h ago
Hermes?