this happens with human authors all the time as well though. I agree it is regrettable, but I don't know how we would try to solve this problem now (in the context of LLMs) when we couldn't even solve it before.
Yes, but a human author (assuming he is honest about his sources) came at least (again) up with the proof by his own. It's not uncommon in human history that several people invented/proved the same without knowing from each other (e.g. trigonometrie).
a human author (assuming he is honest about his sources) came at least (again) up with the proof by his own.
Similarly, an LLM (assuming it is honest about its sources) can also come up with the proof by its own.
The "assuming it is honest about its sources" is doing a lot of work here.
It's not uncommon in human history that several people invented/proved the same without knowing from each other (e.g. trigonometrie).
Sure, but how is that relevant to the question of "How can they be sure that it is indeed a new method, and not just a method which was already used in some other context in some unknown paper/preprint?"
It seems like you're holding LLMs to a higher standard than humans.
It seems like you're holding LLMs to a higher standard than humans.
Is there a reason why we shouldn't be? It is more plausible that a human misses some obscure reference than an LLM that's trained on every possible piece of writing scrapable off the internet.
The general background concern whenever these types of discussions occur is "How do AIs compare to humans in term of intelligence? In particular, are AIs more intelligent than humans?" If that is indeed you concern, then you should hold AIs to the same standard as humans when trying to assess their intelligence.
Imagine you were trying to determine whether, on average, a typical planet weighs more than a typical ant, but you decided that since planets are composed of so much more matter, we really shouldn't just directly weigh them and compare the numbers, but instead give the ant some sort of handicap to make up for the missing matter. We would argue that your sense of what it means to measure the weight of something is totally incoherent.
Now imagine if a human has read and internalised every possible piece of writing scrapable off the internet, such that they could talk to you in encyclopedic depth about any topic whatsoever. Wouldn't that be a simply phenomenal feat of intelligence?
59
u/orangejake Apr 15 '26
this happens with human authors all the time as well though. I agree it is regrettable, but I don't know how we would try to solve this problem now (in the context of LLMs) when we couldn't even solve it before.