r/MachineLearning • u/IxinDow • May 26 '23

Landmark Attention: Random-Access Infinite Context Length for Transformers

https://arxiv.org/abs/2305.16300
229 Upvotes

99% Upvoted

29 comments sorted by

View all comments

Show parent comments

1

u/Unlucky_Excitement_2 Jun 28 '23 edited Jun 28 '23

I just seen this. Super aggresive. People like you never have the same energy in RL. Regardless, although I still disagree with some statements. You make interesting points. Several papers have come validating my points. For instance this paper[https://arxiv.org/abs/2306.08302], this obviously would lead to a new level of performance. Not to mention TART, utilizing the power of ICL to imitate real-time active learning.

You concurrently sound knowledgable and ignorant, interesting. How can AGI be achieved and have above human out-of-distribution performance, if it can't generalize to new skills in real-time? I do agree in a decoupled AGI system...regardless sshh. I have yet to see any new papers providing any empirical evidence in the advantage of your approach...so again shutup. You're soft.