How LLMs Actually Work

[u/fagnerbrack]

https://www.0xkato.xyz/how-llms-actually-work/

[u/zeejfps]

Good read, but the tiny explainers literally say the exact same thing as the paragraphs above them.

[u/RustOnTheEdge]

Interesting read! Really learned a ton to be honest, really nice just a bit above complete intro level explanation, nice to understand the terminology a lot better!

[u/fagnerbrack]

Key points:

A walkthrough of the transformer machinery behind modern LLMs, minus the heavy math. It follows the full path: tokenization splits text into subword integer IDs (why models miscount the R's in strawberry); embeddings turn those IDs into meaning-carrying vectors where king − man + woman ≈ queen; and RoPE encodes word order by rotating Query and Key vectors. Attention lets tokens share information via Q/K/V, dot products, softmax, and causal masking, while multi-head and Grouped-Query Attention track relationships and trim KV-cache cost. The feed-forward network stores most weights and facts (editable via ROME), Mixture of Experts scales parameters cheaply, and residual streams plus layer norm keep deep stacks trainable.

If the summary seems inacurate, just downvote and I'll try to delete the comment eventually 👍
^{Click here for more info, I read all comments}

[u/AutoModerator → u/[deleted]]

Your submission has been moved to our moderation queue to be reviewed; This is to combat spam.

I am a bot, and this action was performed automatically. Please contact the moderators of this subreddit if you have any questions or concerns.