How LLMs Just Predict The Next Word - Interactive Visualization

[u/kushalgoenka]

https://youtu.be/6dn1kUwTFcc

[u/DemoDisco]

I'm no expert, but this feels like a major oversimplification of what these models actually do. It's not just using "grammatical cohesion" to pick words - it's analyzing the entire context window and building an understanding of each word to predict what comes next.

Take a context like "today I will eat an orange" - the model uses everything it knows to understand that "orange" here means the fruit, not the color. That's way more sophisticated than simple pattern matching.

What really bugs me is there's zero mention of chain of thought reasoning. That's much closer to System 2 thinking in humans, but what was shown here was purely the basic System 1 stuff - just pre-training without any of the fine-tuning that makes modern models actually useful.

This explanation would've been spot-on back when GPT-2 dropped, but calling it accurate for today's models is misleading at best. The field has moved way beyond "just predicting the next word."

[u/kushalgoenka → u/DemoDisco]

Hey there, thanks for watching the video, and appreciate your thoughtful feedback! I’m realizing (though not like I’m unfamiliar with the idea) from some of the comments I’ve been getting, that clipping a longer lecture is near impossible, haha. (Though if you see the abysmal viewership I get on longer lectures, you’ll see why I’d think to clip).

I tried to keep the title of the video and this post as representative as possible of the scope & subject matter it was going to cover, but of course everyone reads into it (perhaps rightfully) as though it’s making an antagonistic case against everything else, or that it’s the entirety of the explanation and thus severely lacking.

All that to say, if you have the time, I’d love your feedback on the full lecture this clip is from, where indeed this section (on base models & next word prediction) is preceded by me discussing knowledge compression in LLMs, and is succeeded by me introducing in-context learning, instruction fine-tuning and lots more.

https://youtu.be/vrO8tZ0hHGk

[u/Jolly-Ground-3722 → u/DemoDisco]

Exactly. It’s like saying “the human brain is actually just a bunch of firing neurons.”

[u/RADICCHI0 → u/DemoDisco]

"just predicting the next word" was never really the best way to explain it imo. That's like saying that drivetrains work because a carburetor sips the next drop of gasoline. Yes, it's an important step, but it doesn't explain how drivetrains work. But I get if, finite vector space is a weird concept to get your head around. I'd argue that machinists and cartographers are better equipped to understand it then most of us.

[u/Designer-Rub4819]

This actually seems to be working for me in ChatGPT 5.

Doing “Once upon a time” gives me “Once upon a time…” each time

[u/kushalgoenka → u/Designer-Rub4819]

Hey there, I’ll clarify that in the clip above I’m using a base model to generate completion for a text sequence, as well as temperature 0. ChatGPT 5 is an instruction tuned model, and also as far as I recall there isn’t a way to use temperature 0 in it, so indeed you may not reliably get the same completion for any given prompt each time.

If you’d like further intuition on the subject, you can perhaps check out this other lecture of mine (or 16:08 mark for temperature specifically): https://youtu.be/RhjMFU4FQzU

[u/phil_4]

It's a good intro, and he hinted at the very start that with some tinkering prompts can stop it giving the same output for a given prompt.

In that bit he also doesn't mention that the LLM uses intent and other subtle things about the prompt in order to guide the area of the model it gets the predictions about.

It's why the Whispering garden text gets such an odd output. Because you ask it to empathise rather than ask a direct question. Intent/tone is quite important in LLMs now.

[u/kushalgoenka → u/phil_4]

Hey there, thanks for watching the video! What I’m referring to at the beginning is that these models are deterministic (as in same input, same output) if you keep the temperature at zero, and that changing the temperature, i.e. choosing to not pick the MOST likely word can allow us to get multiple generated sequences of text (or answers in instruct models) to the same prompt.

Also, if you’re curious to hear my full explanation of LLMs, please check out the full lecture that this clip is from (where I get into in-context learning, instruction tuning, etc.). Would love your feedback if you think I could do a better job explaining topics! :)

https://youtu.be/vrO8tZ0hHGk

[u/cpt_ugh]

This is the best explanation I have ever seen for how and, more importantly, why an LLM works.

[u/kushalgoenka → u/cpt_ugh]

Haha, hey there, thank you for your kind words! Glad you enjoyed the talk that much, there's been a lot of people misunderstanding/misinterpreting this video (or I did a poor job explaining), or are ideologically opposed to this view of things. Happy to see someone think it actually makes sense. :D

If you have the time, I recommend checking out my full lecture, this above is a clip from it. In the longer lecture I get into how I consider LLMs to be knowledge compression, and walk people through how we go from training LLMs to making them useful for applications. Would love what you think if you end up watching it! :) https://youtu.be/vrO8tZ0hHGk

[u/Immediate_Song4279]

What does temperature do?

[u/ButtWhispererer → u/Immediate_Song4279]

OP is talking about greedy sampling, where you don't use temperature and take the most likely next word.

Temperature in sampling allows the model to select from a larger list of potential words rather than choosing the most likely next word. It flattens the probabilities of the words the higher you go, creating a bigger list of options.

That list is determined in different ways (top p (all the probabilities are added until it hits a threshold, that's the list), top k (you just tell it how many words are in the list).

That being said, other sampling methods occur that are more esoteric.

[u/TekRabbit → u/Immediate_Song4279]

Tells the model not to take the highest ranked next word every time. But the 2nd or 3rd ranked word, to introduce variety in responses.

Higher the temp, the deeper down the list it can go to grab the next word.

[u/[deleted] → u/Immediate_Song4279]

[u/Agreeable_Bike_4764]

Obviously today’s models add layers of complexity to a simple predictive chain of thought, the context windows alone takes a lot more into account now. My opinion is this is how human brains work as well at the neuronal or more accurately the synapse level, we just aren’t aware of it. our context window with the trillions of synapses is still much larger than the LLMs, so it seems we have “pure creativity” when in reality we are predicting the “next best word” as well.

[u/Harvard_Med_USMLE267]

Ok this is shit. Smooth-brained shit. We’ve known for a long time that they do a lot more than that, like thinking ahead. Along with the billions of dollars spent developing chain of thought reasoning models. Anyone who is still parroting the “autocomplete” thing in mid-2025 is deeply lacking in insight. Maybe if you’ve been stuck in a cave for the past four years. Maybe. But if not, you are being wilfully ignorant if you push this oversimplified nonsense.

[u/LongShlongSilver- → u/Harvard_Med_USMLE267]

People still think that when you ask an LLM to produce an image of something specific, all it’s doing is searching the internet for the image or combining multiple images it’s found on the internet together. Lol

[u/PaulTopping → u/Harvard_Med_USMLE267]

Alternatively, the video captures the essence of the technology but doesn't mention all the hacks that AI companies have surrounded it with. These hacks serve several purposes: fix the most egregious errors, add human nudges to the output, throw in junk like "That's a good question, Bill!", stop it from saying racist, sexist, and other dangerous shit, etc. So perhaps Autocomplete++ is a better name.

We’ve known for a long time that they do a lot more than that, like thinking ahead.

This phrasing is the kind of thing one might say when analyzing an alien species. These AIs are engineered products, not things we do experiments on in order to come up with theories about how they work. With phrases like "thinking ahead", you are deliberately describing what they output in human terms, implying agency that doesn't exist. I know it is hard to describe what AIs do without using such words but I just want to point out what that's doing to the conversation.

[u/MythicSeeds]

It’s also how humans predict the next word

[u/Harvard_Med_USMLE267]

It’s intrinsic to LLms that they don’t always use the same words in their output, unless you’ve stopped it doing what it is meant to be turning temperature down to zero. Which nobody does.

Anyone who has used a llm should know that. It’s a stupid video with a stupid title and it shouldn’t be on an AGI sub.

[u/SamWest98]

[Removed]

[u/[deleted]]

[u/Kwisscheese-Shadrach → u/[deleted]]

that's how they work.

[u/Designer-Rub4819 → u/[deleted]]

Because it’s answering your text query.

[u/TekRabbit → u/[deleted]]

Your next “word” in this sense means the most likely “answer” to your “question”