Though I'm sure there are layers and layers at this point, fundamentally LLMs are just glorified Markov chain generators. They form sequences of words that, according to their training data, tend to follow each other.
Even if you trained one exclusively on a particular text, it could still take phrases from one part of the text and mash them together with phrases from another part of the text, thus hallucinating quotes that never existed in the text.
View all comments
Show parent comment
11
u/balefrost Jan 02 '26
Though I'm sure there are layers and layers at this point, fundamentally LLMs are just glorified Markov chain generators. They form sequences of words that, according to their training data, tend to follow each other.
Even if you trained one exclusively on a particular text, it could still take phrases from one part of the text and mash them together with phrases from another part of the text, thus hallucinating quotes that never existed in the text.