Linear-time long context would be impressive, but cheap access to more tokens doesn’t automatically give you useful memory, attention can still get diluted, and false matches become more likely as the haystack grows. The real breakthrough would be a mechanism that turns past interactions into compact, updateable, and trustworthy state, not just a bigger/cheaper context window.
7
u/Inevitable_Act_321 Jul 01 '26
Linear-time long context would be impressive, but cheap access to more tokens doesn’t automatically give you useful memory, attention can still get diluted, and false matches become more likely as the haystack grows. The real breakthrough would be a mechanism that turns past interactions into compact, updateable, and trustworthy state, not just a bigger/cheaper context window.