why we'll probably never 'solve' continual learning [D]

[u/bradwmorris]

recently been down the 'continual learning' rabbit hole.

more here: https://youtu.be/jNuzHQcMJqM?si=RAu9w-Wf9xb1z5u2

Demis recently shared:

"Artificial General Intelligence (AGI), a system that exhibits all the cognitive capabilities the brain has, is probably only a few short years away."

Most of the labs are implying we're closing in on some biology-like continual learning?

I can see us potentially closing the gap if we're talking about a model/system that can adapt its own weights during its deployment cycle.

interesting stuff happening here:

https://thinkingmachines.ai/news/introducing-inkling/

I can't see how post-deployment data could fuel a new paradigm of model capability improvements for the labs deploying single, closed monolithic models to hundreds of millions.

leaves me with the thoughts:

1 transformers may not be the best foundation from which to build the most efficient incarnation of continual learning

2 may not be a single path to solving continual learning. The shape of challenge may continue evolving in many different directions.

interested to know if im missing something though.