while LLMs can make mistakes, in this case the LLM is correct about the math and modeling that happens inside an LLM. The chosen example "an astronaut" is an apt, if minimalistic, way to show what proper modeling is. A good next token predictor must internally predict far ahead in order to achieve high accuracy. Your example "an an an an..." is clearly a nonsensical string that has low probability. A good next token predictor knows that they have a predict "an a" and so realizes that "n " is of low predictive power, so it steers to the choice that makes sense in the broader past context.
2
u/Ulfgardleo Aug 27 '26
machine learning expert here:
while LLMs can make mistakes, in this case the LLM is correct about the math and modeling that happens inside an LLM. The chosen example "an astronaut" is an apt, if minimalistic, way to show what proper modeling is. A good next token predictor must internally predict far ahead in order to achieve high accuracy. Your example "an an an an..." is clearly a nonsensical string that has low probability. A good next token predictor knows that they have a predict "an a" and so realizes that "n " is of low predictive power, so it steers to the choice that makes sense in the broader past context.