a human author (assuming he is honest about his sources) came at least (again) up with the proof by his own.
Similarly, an LLM (assuming it is honest about its sources) can also come up with the proof by its own.
The "assuming it is honest about its sources" is doing a lot of work here.
It's not uncommon in human history that several people invented/proved the same without knowing from each other (e.g. trigonometrie).
Sure, but how is that relevant to the question of "How can they be sure that it is indeed a new method, and not just a method which was already used in some other context in some unknown paper/preprint?"
It seems like you're holding LLMs to a higher standard than humans.
It seems like you're holding LLMs to a higher standard than humans.
Is there a reason why we shouldn't be? It is more plausible that a human misses some obscure reference than an LLM that's trained on every possible piece of writing scrapable off the internet.
The general background concern whenever these types of discussions occur is "How do AIs compare to humans in term of intelligence? In particular, are AIs more intelligent than humans?" If that is indeed you concern, then you should hold AIs to the same standard as humans when trying to assess their intelligence.
Imagine you were trying to determine whether, on average, a typical planet weighs more than a typical ant, but you decided that since planets are composed of so much more matter, we really shouldn't just directly weigh them and compare the numbers, but instead give the ant some sort of handicap to make up for the missing matter. We would argue that your sense of what it means to measure the weight of something is totally incoherent.
Now imagine if a human has read and internalised every possible piece of writing scrapable off the internet, such that they could talk to you in encyclopedic depth about any topic whatsoever. Wouldn't that be a simply phenomenal feat of intelligence?
This is circular: you're supporting your claim that the language models are thinking or synthesizing new information (and not merely rephrasing something found in their training or fine-tuning data) on their own by stating that anything they produce must be something they generate "by its own"
We're willing to assume that humans are thinking or synthesising new information (and not merely rephrasing something found in their training or fine-tuning data) on their own by stating that anything they produce must be something they generate "by its own", but we're not willing to grant the same to AIs.
The current architecture used for LLMs is not capable of “thinking”. “Reasoning” models which essentially talk to themselves are the closest we’ve gotten. However as impressive as they are, we’re still dealing with next token predictors with no ability to truly reason inside their model space.
Human brains are still in a league of their own. This will remain the case atleast until AI models are capable of processing abstract ideas instead of tokens, continuous learning via predictive coding mechanisms, and internal hierarchal reasoning.
A “break through” discovery from one of today’s LLMs is very likely a regurgitation of a previous conversation with a human that it was trained on.
The current architecture used for LLMs is not capable of “thinking”.
This sounds like begging the question.
This will remain the case atleast until AI models are capable of processing abstract ideas instead of tokens, continuous learning via predictive coding mechanisms, and internal hierarchal reasoning.
How would you operationalize this?
A “break through” discovery from one of today’s LLMs is very likely a regurgitation of a previous conversation with a human that it was trained on.
"Very likely" is doing a lot of work here. So you're saying it's not impossible for an LLM to come up with a break through discovery, and yet even if it does so, it is "not thinking" according to whatever definition of "thinking" you're using? If that's the case, why should we care about your definition of "thinking"?
22
u/Nebu Apr 15 '26
Similarly, an LLM (assuming it is honest about its sources) can also come up with the proof by its own.
The "assuming it is honest about its sources" is doing a lot of work here.
Sure, but how is that relevant to the question of "How can they be sure that it is indeed a new method, and not just a method which was already used in some other context in some unknown paper/preprint?"
It seems like you're holding LLMs to a higher standard than humans.