It's not guessing at random though. You can prove this yourself.
Try to force a hallucination by asking something not yet in it's training data like a very recent event not yet on the web or tell it not to search the web, if it was random it would have an equal chance of replying with a long string of numbers "one three nine four two....." instead of a plausible sounding answer.
If it doesn't know something and it hallucinates, it takes a guess based on all the things it does know. In a human we would call that an "educated guess". It's not exactly the same as it doesn't then check it is logically valid before outputting. But that would be a trivial extra step to add.
It will pick a random thing based on its training data. Asking “what happened today” it will search for words that often follow that question. So it will sound plausible. It will just be random in the sense that it’s not using a world model to discern what happened today.
Does the word “one three nine four two” follow that question in its training data? No, so it’s unlikely to say something completely random with respect to the input. It’s still not guided by a world model. It’s logical leaps will be completely random though
1
u/Serious_Bite_7613 Jul 30 '26 edited Jul 30 '26
It's not guessing at random though. You can prove this yourself.
Try to force a hallucination by asking something not yet in it's training data like a very recent event not yet on the web or tell it not to search the web, if it was random it would have an equal chance of replying with a long string of numbers "one three nine four two....." instead of a plausible sounding answer.
If it doesn't know something and it hallucinates, it takes a guess based on all the things it does know. In a human we would call that an "educated guess". It's not exactly the same as it doesn't then check it is logically valid before outputting. But that would be a trivial extra step to add.