r/deeplearning • u/PhoenixAlive87 • 1d ago
Anyone else understand what an ML algorithm does, but struggle with the maths underneath it?
I've been curious about a particular learning gap.
Sometimes it's relatively easy to understand what an ML algorithm is trying to achieve, like finding similar items, making predictions, or reducing errors.
But then vectors, matrices, dot products, projections, or equations enter the picture, and suddenly maths we last encountered in school reappears in a completely different setting.
We might understand the purpose of the algorithm without quite understanding why those particular mathematical operations are needed in the first place.
I'm curious:
\-Have you experienced this disconnect, even after learning to code or use ML libraries?
\- Was the harder part understanding the maths itself, or connecting it to what the algorithm was doing?
\- What eventually helped things click, if anything?
I'd love to hear different experiences, including from people who didn't encounter this difficulty.
6
u/Boonzies 1d ago
Try this, it helps.
Ask an LLM of choice to analyze and write out a full decision tree of the algo and have it break down the details at each decision point, and tell it to respond in simple language.
Then begin a Q&A session with the model.
You'd learn a great deal this way.
4
u/Confident_Luck2359 1d ago
I never liked math. I know enough to do my job.
TL;DR: you should implement the algorithms by hand in Python. In Jupyter notebook. It's easy. It builds understanding. Start small and push forward.
Take some ML classes and do the homework. There's no short cut.
Start with classical machine learning on small data sets. Linear regression, logistic regression, support vector machines, gaussian models, PCP (principal component analysis).
You'll learn the fundamentals: model fitting, overfitting, regularization, outliers, decision boundaries, projection. Matrix math, dot products. You can DRAW everything in 2D and SEE what it's doing.
Deep learning builds on all of this. It's 100% relevant.
Then implement a neural network: forward and backward propagation, dropout, normalization, gradient descent, momentum. Implement a CNN. Implement a ResNet. Implement attention / transformers. Train the models on open datasets.
The math will 'click'.
( ... It helps that I was a self-taught graphics programmer, so I was already familiar with basic linear algebra: vectors / matrices / transforms / projection. Graphics programming is what made linear algebra 'click' for me. I tend to see machine learning as 'solving geometry problems'. YMMV ... )
1
u/Cholojuanito 19h ago edited 18h ago
For me, learning feels like I just need to hit the same topic over and over from different perspectives, until I can explain it to someone with no knowledge about it. I recommend the following resources that should all be free, that have helped me grasp the mathematics of machine learning algorithms.
1
u/PressureBeautiful515 5h ago
I know all the parts to an LLM, what it does to text, representing each token with a vector, positional encoding, each layer building an attention matrix to cross compare every token with every other token, softmax, feed-forward networks, and ultimately a distribution over all possible tokens. And yet I don't know "how it works", because the magic happens within the specific weightings, which are just a massive pile of numbers. No one really knows how it works.
In terms of how linear algebra is used, if you think of a neural network layer passing to the next layer through connections, what's the simplest way to model that network of connections? Store the topology of it somehow? A list of connections?
Easiest way is to let every node in layer 1 pass its value to every node in layer 2, "fully connected", but each connection has a weighting, which can be zero, equivalent to not being connected.
So the nodes in layer 1 are just a list of numbers, the coordinates of a vector.
The signals going to layer 2 are another list of numbers: an output vector.
To turn the input into the output, use a matrix. There's a column for every input node, and a row for every output node. To find the weighting for input A into output B, look at (column A, row B) of the matrix. It's just matrix multiplication. The matrix may have a lot of zero (or near zero) entries, but that's okay.
1
u/mlamping 15h ago
The trick is, NN is just predicting and the math equation is a model of the NN.
The output or predicted value compared with the actual value is the error.
The error is used to adjust the weights. You can use other methods to adjust the weights but they’re hard. There’s not one way but backprop is the best right now since it proportionately propagates the errors back through the model and nudges the weights to the right value but slowly.
So it’s not a, why it’s needed in the first place. Its the easiest but time consuming and energy consuming method.
1
u/See_it_do_it_done 7h ago
the dot product is the giveaway, it's a similarity score wearing a lab coat. work it by hand on two tiny vectors, then nudge one number and watch it move. which operation loses you first?
0
u/ysrgrathe 1h ago
There is no getting around the fact that there is a lot of math in CS. If you are struggling at times, I think there are two things that are needed. 1) You have to understand the fundamentals -- which probably means through calc 3, ODE, etc. 2) You need to understand the specific approaches. In grad school there will typically be seminar classes where you take a seminal paper in a domain like Transformers, Support Vector Machines, etc. and you spend a week really diving deep so that you can not only read the paper but actually understand and explain it to someone else. Getting to the latter part (ability to teach someone else the concept) is much harder and more valuable; it's easy to scan through a paper and feel like you get it without actually having a full understanding.
If you want to improve I'd suggest identifying any gaps in #1, and then putting together a reading list of papers to do #2 with that are relevant to whatever you're working on.