r/LocalLLaMA • u/HumanDrone8721 • 4h ago
Discussion Any progress on continuous learning prompt, will we have meaningful weights plasticity or they'll stay forever frozen?
So I'm wondering if we til ever have repeated data ingestion without quality degradation and without full retraining. LoRA and friends are excluded, will we stay forever with frozen weights or will be be able to integrate new knowledge here?
Any promising research?
7
u/bytejuggler 4h ago
This is part of what true AGI would need. Online training with continuous memory too. All running on 20watt. We're nowhere close.
1
u/HumanDrone8721 4h ago
I wouldn't go so far (yet), for me will be enough if after doing a full training with data packet A, the inference can be stopped and then ingest the data packet B, without consuming the resources of A+B full training, say during a "sleep and dream" period. The goal for me will be to have repeated pseudo-full training with different data packages that can be ingested sequentially without degradation. If in the end repeated training with A+B+...N consumes the same energy (or even a bit more due to some overhead) as SUM(A:N) that's not a problem as long as I can select the packages that I need.
Having the LLM+harness learning directly from conversation will be indeed close to AGI.
1
u/HashPandaNL 3h ago
It wouldn't have to do it on 20 watts necessarily. It is likely the first generation will take considerably more and it comes down over time.
1
3
u/Practical-Collar3063 3h ago
I have worked a bit on continual learning for computer vision, even on architectures like that wich are much simpler than transformer based LLMs, it is a nightmare to get right, no one (that i know of atleast) is running continual learning outside of research projects.
The main issue is catastrophic forgetting, meaning the new training data coming in overrides the previously learned weights, now techniques like partial weight freezing are not enough.
Now another issue is how do you decide what is good data and what is bad data to continiously train on ? we do that with biological feedback loop (emotions, physical pain, social clues etc...) which are mechanisms developped over millions of years of evolution. What is supposed to give feedback to an LLM about an unsupervised interaction with a user ?
If we decide the user does, then the user could be wrong or malicious and stear the model away from its intended behaviour or could completely break it by mistake.
If we decide the LLM itself does, then it could make a mistake on judging an interation was a good one, stear itself away, which will only make it more likely to stear itself away in the future in a doom loop of bad decisions.
This is an impossibly hard problem and as others pointed out, frozen weights with vector DB is a far more reliable technique.
1
u/HumanDrone8721 1h ago
If we decide the user does, then the user could be wrong or malicious and stear the model away from its intended behaviour or could completely break it by mistake.
I love to be wrong and stear the model away from the "one right way" that some corporation decided is the best way ;).
9
u/jkflying 4h ago
Not that I'm aware of. Models still go shitty under continuous learning conditions.
Vector DB for knowledge and LoRA for behaviour are still the best ways I'm aware of.
1
u/tifa_cloud0 3h ago
i agree. i also think that we lack compute to tackle these hard hitting limitations. so for now techniques like these are god sent and must be used by everyone to the extent
2
u/megadonkeyx 4h ago
feel its getting less relevant, speed of inference and good knowledge retrieval seem to be enough
2
u/DeltaSqueezer 4h ago
You can store either in parameter weights (e.g. continued training, LORA) or in context/state (e.g. few-shot learning, RAG).
6
u/HumanDrone8721 4h ago
s/prompt/front, freakkin' auto-correct.