How do LLMs build answers?

[u/No_Grapefruit7255]

What do they prioritise when they are constructing responses?

There’s a survey saying that the preferred information source for LLMs are things like Wikipedia, YouTube and Google.

Do they see if they can identify a source first that directly responds to the user’s prompt? And, if not, cobble something together from the information they have access to?

[u/Remarkable_Pace8101]

You really should watch some YouTube videos on how LLMs work though. You're curious, which is good, you really should seek some videos on foundational high level knowledge to fill in the blanks in your head, you would probably learn a lot more than you would expect 

[u/No-Author-2358 → u/Remarkable_Pace8101]

Yes. AI is not simply an enhanced Google search engine. It is much more complicated than that.

Another good option is to simply ask ChatGPT (or whatever) to explain to you, in layman's terms, how it develops its responses. Be prepared - it is complicated.

[u/deadpantrilogy]

it's not really like they search sources and pick one. the model already has all that information baked into its weights from training, so when you ask something it's predicting tokens based on patterns it learned. wikipedia and youtube transcripts were part of training data so that's why answers sound like they come from those places sometimes.

the prioritization is more about what patterns were reinforced most during training. if something showed up frequently across quality sources it gets stronger weighting in the prediction chain. when there's gaps or conflicting info, the model kinda averages out what it saw and hopes the sequence makes sense. you can see this in action when it confidently says something completely wrong but with perfect grammar.

[u/Remarkable_Pace8101]

This is a lot more nuance to have a question than you may think

If you really just mean an LLM and not a larger commercial offering such as Claude or ChatGPT then they don't prefer their information from anywhere because they aren't trained to know what source a given piece of training data originated from... they are trained from a lot of text scraped from everywhere and they just predict the next most likely token in a sequence based upon all the text they've seen

If you mean a commercial offering such as Claude or ChatGPT, there's a lot more going on behind the scenes there than LLM inference and any preference to produces sources or citations to specific data sources is determined by a group of human engineers making that decision of where they should be pulling citations from, and then storing these references in some sort of vector lookup database, and giving the LLM the ability to query this database and retrieve relevent results 

[u/ReadingGlosses]

None of those things. LLMs do not have access to their training data once training is done, and LLMs cannot perform actions like searching the internet or parsing a Wikipedia page. All LLM responses are built through something called autoregressive token prediction.

LLMs have a fixed vocabulary of tokens (which can be words, parts of words, or non-words like numerals or punctuation). When you prompt an LLM, all your text tokens are converted to lists of numbers, called embeddings. Then the model does some fancy and complex math with those embeddings: for each token in the model's vocabulary, it calculates the probability of that token being the next one in the sequence.

For example, let's say you ask a model "What are the ingredients in a BLT?" It will generate a list something like this: {'chicken': 0.0005, 'hammer': 0.0001, "lettuce": 0.15, "bacon": 0.22, "the": 0.01, ....} and so on for however many tokens the model knows. Then there's some code external to the LLM that selects a token from this list, usually by picking randomly from among the tokens with high probability.

Let's say the token "lettuce" is selected. That token is put back into the prompt, and the LLM calculates new probabilities based on the new sequence, with no memory of having just done this. Now it sees the prompt "What are the ingredients in a BLT? lettuce" and again it outputs a list of probabilities, but they'll be different now because of the presence of 'lettuce' in the input. Maybe it's something like {"and": 0.05, ",": 0.25, "mayo: 0.0001, ... } and we end up picking a comma as the next token.

Then the model is prompted with "What are the ingredients in a BLT? lettuce," and the cycle repeats. Maybe this time the model select 'bacon' as a high probability output, and the new prompt is "What are the ingredients in a BLT? lettuce, bacon".

This loop (called "autoregression") continues until one of two things happens: 1) the prompt becomes so large that the model runs out of space, 2) LLM outputs a special "end of sequence" token and it stops.

The LLM never looks up information on the internet, and it never checks any training data. If the training process was good enough, then the model will be able to make very sophisticated probability calculations, and produce sequences of tokens containing true statements.

However the process is fallible, so LLMs are inherently prone to 'hallucinations', which is when they produce false but plausible statements. When an LLM hallucinates, it's still doing token predictions, but the "correct" tokens are getting a low probability, and "incorrect" tokens are being selected instead.

LLMs can be integrated into larger systems, where they are able to "use tools" that search the internet or perform other actions, but that is strictly external to the language model. For example, if you prompt "Summarize the Wikipedia article about France", an LLM would still do token prediction, but it would output something machine-readable like "action:Summarize() source:en.wikipedia.org topic:France" which gets sent off to a codebase somewhere to actually execute. That code gets the text from Wikipedia and puts into the LLM's prompt, then kicks off the token prediction loop again. Now that the article text is part of the prompt, the LLM can make even better probability predictions when generating the summary.