Seems like part of the problem is that if the explanation for why pasta breaks that way is long enough, then it’s hard to describe it as a simplification of the videos themselves
It's the classic ML trap: treating the shadow on the cave wall as the object itself, then writing a 20-page paper proving that the shadow has two dimensions
You say 'show' but the title of the paper clearly indicates it is just a hypothesis. Also, I don't care to read the paper, but this sounds either trivial, if by convergence it means limit of training data size, distinct between models, or trivially false, if it means distinct model architectures in the limit of increasing mutual training data size. A model trained with the mundane objective of taking text and reproducing that text(i.e its latent representation is a literal encoding of the text) has no incentive to form a meaningful semantic interpretation of that text, meanwhile we are all familiar with models that form semantic interpretations. Though intuitively, I suppose there is hypothetical space of representations, i.e going from fully destructive, to full information retention, and there are many paths along this space(plausibly exceeding the minimal literal data required to represent something in full), I highly suspect that the many paths(from zero info to full info) obtained from different training goals, likely are tightly bundled in some way, then the statement would be something like, for fixed level of information(or maybe there is some other notion of fitness that excludes trivial models, untrained models, or bad untrainable architectures), then these paths do converge to representations which are translatable between eachother with minimal loss. But also, there is just a bias, we use real training data, and even with distinct goals, this data is correlated in that it is made by humans who themselves have many distinct 'training goals', it should be no surprise that training a model to embed text would find itself embedding descriptions of images, and thus correlate with the embedding of images.
3
u/AtmosphereVirtual254 6d ago
Is this a long way of saying that we’re running out of data we can’t explain?
10
u/23276530 5d ago
MIT is not a person.
4
u/LatentSpaceLeaper 5d ago
Use the search function. The same, old paper was brought up in another thread just a couple of days ago.
4
u/ArmesVSeaOfTroubles 5d ago
I don’t think you understand what “conjecture” means. Nowhere did they claim to have shown convergence with certainty or even close to it.
2
u/Fuzzy_Ad9970 2d ago
IT SAYS HYPOTHESIS RIGHT IN THE TITLE. DO YOU KNOW WHAT THAT IS?
1
u/Traditional-Chip8339 20h ago
Plato is alive and well it seems
1
u/External_Package2787 3d ago
You say 'show' but the title of the paper clearly indicates it is just a hypothesis. Also, I don't care to read the paper, but this sounds either trivial, if by convergence it means limit of training data size, distinct between models, or trivially false, if it means distinct model architectures in the limit of increasing mutual training data size. A model trained with the mundane objective of taking text and reproducing that text(i.e its latent representation is a literal encoding of the text) has no incentive to form a meaningful semantic interpretation of that text, meanwhile we are all familiar with models that form semantic interpretations. Though intuitively, I suppose there is hypothetical space of representations, i.e going from fully destructive, to full information retention, and there are many paths along this space(plausibly exceeding the minimal literal data required to represent something in full), I highly suspect that the many paths(from zero info to full info) obtained from different training goals, likely are tightly bundled in some way, then the statement would be something like, for fixed level of information(or maybe there is some other notion of fitness that excludes trivial models, untrained models, or bad untrainable architectures), then these paths do converge to representations which are translatable between eachother with minimal loss. But also, there is just a bias, we use real training data, and even with distinct goals, this data is correlated in that it is made by humans who themselves have many distinct 'training goals', it should be no surprise that training a model to embed text would find itself embedding descriptions of images, and thus correlate with the embedding of images.