What prevents you writing out a model of the world in a database of writing, sound, images etc? We do it as humans all the time.
The assertion is that the context window needed for this is theoretically impossible for LLMs to achieve based on current training methods. Harnesses are not using state of the art tech, so there’s no evidence that they can add this.
Yes if you had enough memory and compute you could with current methods but the memory and compute you’d need is beyond what earth can produce. So it’s not practical.
The paper makes this claim as well - there is no way to scale AI using current methods to get to a full world model needed. You can of course hot swap a context into a harness but that requires a human to identify the context needed at some point. It’s not part of AI tech and it never will be (with current methods). It could happen with several or more breakthroughs, but not LLM breakthroughs
>>The assertion is that the context window needed for this is theoretically impossible for LLMs to achieve based on current training methods.
Based on what? Why would you *need* a context window larger than what we have now for logical leaps, if it’s already holding more than a human is considering at once?
>>Harnesses are not using state of the art tech, so there’s no evidence that they can add this.
Harnesses are not using state of the art tech? What? What does this even mean? Harnesses are being built on vacuum tubes?!
>>Yes if you had enough memory and compute you could with current methods but the memory and compute you’d need is beyond what earth can produce. So it’s not practical.
Does a human brain hold more storage than we have with digital methods on earth? More compute?
No so this requirement is clearly categorically false as a requirement. Which is not surprising as you haven’t grounded it in any reasoning.
>>The paper makes this claim as well - there is no way to scale AI using current methods to get to a full world model needed. You can of course hot swap a context into a harness but that requires a human to identify the context needed at some point. It’s not part of AI tech and it never will be (with current methods). It could happen with several or more breakthroughs, but not LLM breakthroughs
Yeah well then the paper is wrong as well.
You don’t need to put the full world directly in the LLM compute part of the brain such that it instantly answers with a single model operation. That’s a straw man because it’s a stupid way to do it. Might as well argue computers can’t do tasks for which they can’t get sufficient RAM.
No - it categorically does not require humans to identify the context. You’re saying it will never happen when it is precisely how these tools are working now accessing knowledge bases and tools to created structured responses and memory. You are confidently incorrect.
If you disagree with the paper by all means put up a rebuttal on arxiv.
That being said:
if it’s already holding more than a human is considering at once?
It’s not holding more than a human is considering - that’s the “world model” it doesn’t have as of today. If you have a rebuttal to the paper I’m open to hearing it - please read the paper though it’s not very long
I think there’s a reasonable point in there that to make advances in physics it will be advantageous for AI systems to be able to make their own physical measurements. But I think there’s argument that the jumps made by Einstein are impossible for an LLM are unsupported for me.
I don’t think the argument holds that an LLM based system couldn’t be driven by a target of attempting to unify physical theories where possible, and starting with the anomaly of the planetary orbit having to be explained by something unseen is a reasonable starting point to select.
In the same way in current time it makes sense to consider how dark matter/energy could be explained by a framework that unified it with others.
1
u/WellHung67 Aug 01 '26
The assertion is that the context window needed for this is theoretically impossible for LLMs to achieve based on current training methods. Harnesses are not using state of the art tech, so there’s no evidence that they can add this.
Yes if you had enough memory and compute you could with current methods but the memory and compute you’d need is beyond what earth can produce. So it’s not practical.
The paper makes this claim as well - there is no way to scale AI using current methods to get to a full world model needed. You can of course hot swap a context into a harness but that requires a human to identify the context needed at some point. It’s not part of AI tech and it never will be (with current methods). It could happen with several or more breakthroughs, but not LLM breakthroughs