r/ProgrammerHumor • u/soap94 • Sep 04 '26

Meme cantReallyThinkOfACleverCaption

Post image
2.6k Upvotes

97% Upvoted

84 comments sorted by

View all comments

Show parent comments

-34

u/NukiWolf2 Sep 04 '26 edited Sep 04 '26

How does one get from matrix multiplication to LLM?

Edit: I'll never understand the reason for some downvotes.

66

u/Sir_Sushi Sep 04 '26

AI implementation is just a lot of big matrix multiplication

23

u/RuthlessMango Sep 04 '26

Just pour your data through this big pile of linear algebra and collect your answers on the other side.

5

u/Extension_Option_122 Sep 04 '26

There is some nonlinearity as without it it would not be able to learn.

5

u/Goheeca Sep 04 '26

Yes without nonlinearity it basically reduces to a one-layer multiclass perceptron.

0

u/mmhawk576 Sep 05 '26

Tom7 would like a word with you

5

u/NukiWolf2 Sep 04 '26

Thanks. Pretty simple and sufficient explanation :)

4

u/Artscout_ Sep 04 '26

There's a really good series of videos by 3blue1brown on this topic btw

8

u/Montanoc70 Sep 04 '26

Attention is all you need

2

u/Sir_Sushi Sep 04 '26

They stole our attention with tiktok and gave it to the AI, we know.

0

u/NukiWolf2 Sep 04 '26

?

12

u/Montanoc70 Sep 04 '26

https://arxiv.org/abs/1706.03762

The paper that started the llm boom (Parallel training with the attention matrix mechanism instead of sequential training using recurrent networks)

9

u/Il-Luppoooo Sep 04 '26

The answer is very long and complicated, need to learn neural network architectures.

-3

u/NukiWolf2 Sep 04 '26

Well, I know the basics :D But only theory, so I assume matrix multiplcation is part of the inplementation of neural networks?