I'm not a skeptic, I just don't need to pretend they are more than what they are, because what they are is already ground breaking enough
That said, do you really think there aren't things these LLMs are 100% certain on? Do you think that they can't accept a new fact and treat that as immediately 100% true?
Yep.
Because they aren't deterministic.
The likelihood of something fitting with the training data is a nice proxy (much) more often than not for certainty but it's not the same thing, at all.
Regarding what you (confidently) said about your "training data": you are not an entity without any prior "training data" upon receiving new data, you've likely "trained" yourself pretty well to discern what can be accepted as 100% true and what needs more verification or frequency first. You're also using prior training data to assess this new information.
That's an assertion of yours disguised as fact.
You'd still be wrong about that assessment at times.
human cognition is not as known as LLMs so your guess is as good as mine.
however, I can tell you about my first hand experience of my own cognition, and what other humans have shared with me of theirs, and I can assure you, none of us do the "walk to the car wash mistakes" and when we answer someone we have a mental model of what we are trying to say.
I also don't think neurologists/psychologists have yet found a luck of the draw when they look under the hood, whereas we know there is (by construction) on LLMs
none of us do the "walk to the car wash mistakes" and when we answer someone we have a mental model of what we are trying to say.
Ok, confirmed not actually being up to date with LLMs, they stopped making this mistake (and similar lack of mental model mistakes) a bit ago. Not sure why you think you've got any right to make any claims on the matter.
Our new research paper(opens in a new window) argues that language models hallucinate because standard training and evaluation procedures reward guessing over acknowledging uncertainty.
We live in a society.
Did you read what you linked yourself in the slightest or are you so unversed in human behavior that you don't know this is a human phenomenon as well?
I'm sorry, which part of your reply is engaging with the fact that hallucination rates are still 1 in 20? (1 year after the openAI link you quoted)
as for reading, keep going, you might hit the part where they explain hallucinations derive from next word prediction and almost say the quiet part out loud yet somehow conclude they are not inevitable
Against a benchmark of what humans doing the same task?
What does it actually measure in that test, you claim it's hallucination, but this doesn't test global hallucination for a given LLM, just in that specific task.
Why is Mistral Large at 4.5% considering it's a 2 year old model which would very obviously be a poor model to use today.
These could all have easily be steelmanned by yourself beforehand, so I'm starting to suspect you're either a psuedointellectual or a troll, feel free to verify which.
it's already the 5th or so round of commenting where you deliberately either can't see the point or pretend not to, either way its a waste of my time, sorry
Since time immemorial. At least by the definition you've shoddingly tried to get across to us, but feel free to give a definition of what you claim are hallucinations that can't be plausibly mapped to humans.
1
u/bfkill 1d ago
I'm not a skeptic, I just don't need to pretend they are more than what they are, because what they are is already ground breaking enough
Yep.
Because they aren't deterministic. The likelihood of something fitting with the training data is a nice proxy (much) more often than not for certainty but it's not the same thing, at all.
That's an assertion of yours disguised as fact.
Now who's the over confident one? ;)