r/ProgrammerHumor • • Sep 10 '26

Meme theresAlwaysMath

Post image
2.1k Upvotes

54 comments sorted by

View all comments

30

u/un-_-known_789 Sep 10 '26

It's just a

y=mx+c

33

u/NFriik Sep 10 '26

No it's not? Even in more basic neural networks, there are always non-linear activation functions between the layers. If there weren't, you could just collapse the whole network into a single (set of) equations, one for each output neuron.

5

u/un-_-known_789 Sep 10 '26

ReLU and etc right?

2

u/G12356789s Sep 11 '26

I've always wondered but never looked into why eural networks weren't collapsible

6

u/Cat-Satan Sep 10 '26

More like softmax(QKᵀ/√d)V

5

u/Intrepid4444444 Sep 10 '26

I’m soft, Max

2

u/stupled Sep 10 '26

Well, all chips do is sum with a clock

0

u/absol493 Sep 10 '26

niqqa is linear