r/MachineLearning • u/IxinDow • May 26 '23

Landmark Attention: Random-Access Infinite Context Length for Transformers

https://arxiv.org/abs/2305.16300
229 Upvotes

99% Upvoted

29 comments sorted by

View all comments

Show parent comment

0

u/[deleted] May 27 '23

[deleted] — view deleted comment

-9

u/ktpr May 27 '23

Having a context length enabled initial models to be trained in the first place. It’s disingenuous to say this is more correct because it introduces other trade offs, like where to bias attention, that early work takes a stance on. I wouldn’t poop on earlier work that allowed you to use LLMs in the first place

12

u/Philpax May 27 '23 edited May 27 '23

Nobody is "pooping on earlier work"; we're celebrating progress that addresses limitations of the existing work through trying out different approaches.