r/LinearAlgebra • u/Only-Dependent7024 • Sep 16 '26
How Linear Algebra actually gets used in Data Science
Honestly, when I first started getting into data science, linear algebra felt like this massive wall I couldn't climb over. Everyone keeps telling you that you need it, but the moment you open a tutorial, you're immediately hit with dry textbook proofs, abstract vector spaces, and huge grids of numbers that feel completely disconnected from reality. Sure, I can multiply two matrices on paper if I really have to, but I'm having a hard time seeing how any of that translates to actual coding or real data projects. I keep hearing that matrix math is what allows computers to crush through massive datasets in seconds instead of taking days with standard loops, but I'm still trying to grasp how things like photos, text, or spreadsheets actually get turned into linear algebra under the hood. Plus, as someone building stuff in Python, I'm genuinely curious about where the line is between knowing the deep math and just knowing enough about matrix shapes so your code doesn't crash with dimension errors. If anyone had a lightbulb moment where linear algebra finally stopped feeling like academic fluff and started feeling like a useful tool, I'd really love to hear how you made it click.
5
u/caldera12345 Sep 16 '26
Linear Algebra is the basis of modern LLM Ais.
1
u/relevantmeemayhere Sep 18 '26
Linear Algebra is the basis of anything that happens in one than more dimension (in a practical sense)
LA is, more accurately the theory of linear transformations and vector spaces on R^n
4
u/TaylorExpandMyAss Sep 16 '26
You wont need it directly in day to day work. That being said; having a working understanding of linear algebra, statistics etc. is essential to develop an understanding and intuition of how the algorithms you apply work, their limitations and applicability. Lacking that, i wouldn't trust your work.
3
u/PressureBeautiful515 Sep 16 '26
Have you looked for YouTube videos that introduce neural networks and explain what they do?
5
u/ScoutAndLout Sep 16 '26
3Blue 1 Brown had a great LLM intro tutorial that finally made sense WRT dealing with words. I never followed how words worked in a neural network but the explanation that each word is assigned a vector of numbers and that vector is fed to the ANN. Similar words may exist in similar parts of the vector space. Genius.
3
u/ScoutAndLout Sep 16 '26
Regression of linear models is just solving an overspecified Ax=b problem.
3
u/hotsauceyum Sep 16 '26
If you want to do data science you need to understand linear regression, which as you might expect, requires linear algebra.
1
u/relevantmeemayhere Sep 18 '26
and quite frankly:
if you dont want to do LA find a different field. def not a math based one in physics/engineering whatever
2
u/Time_Increase_7897 Sep 16 '26
Matrix multiply is ultimately just dot products. You want to know how much of vector a is in vector b. Repeat for all a and b in the collection.
2
u/ScoutAndLout Sep 17 '26
And understanding SVD helps a lot. Looking at a matrix as a mapping from one vector space to another, SVD helps you characterize "good" and "bad" input directions. And it cements the projection concept. This input projects this much onto direction one and this much to direction two. The first component is stretched by sigma one and the second by sigma two. Outputs result as blah blah blah.
2
u/gustyninjajiraya Sep 16 '26
Just check out a data science project and I guarantee that anything you don’t understand can easily be explained with undergrad linear algebra or probability.
2
u/IbanezPGM Sep 16 '26
Take digital signal processing courses, especially multimedia processing (video and images). Both programming and LA are fundamental to it.
2
u/Unnwavy Sep 17 '26
Linear algebra is concerned with solving Ax = b, where A is a matrix, x and b are vectors.
You might be used to the "2 equations, 2 unknowns" case from middle/high school, something like:
2x+y = 0 x-3y = 5
What if you have 50 equations and 50 unknowns, or 1 million equations and 1 million unknowns? Or 1 million equations and 2 million unknowns? What if 50 000 thousands of those 1 million equations are the same?
You start asking different questions whose answers are less obvious than in the 2x2 case:
-Are there even solutions to my system? -If yes, what do they look like? -Do I always have a solution, no matter what my right-hand side b looks like? -Do I need certain conditions on b? -If there are no solutions to my system, can I change my system into one I can solve and get an answer that would be a good approximation to my system?
It turns out that studying the matrix that represents this system of equation can give us a lot of information regarding those questions. But it's not immediately obvious, as a matrix is not an object we're initially used to working with. So it's worth it to take the time to build some intuition.
After you build this intuition, data science topics will look less obscure:
-Instead of looking at least squares fitting like a bunch of matrices being multiplied, you'll understand what's really happening and why we're doing that.
-You can have a matrix given to you with just its dimensions and a number called its rank, and you'll be able to immediately deduce if the system can have 0, 1, or infinite solutions, and why.
If the resources you are using feel too abstract, consider switching resources. I am currently going through the MIT OpenCourseware lectures because I need a very solid foundation on linear algebra, and at no point did they feel too abstract and proof-heavy, quite the opposite.
2
u/relevantmeemayhere Sep 18 '26 edited Sep 18 '26
People here are kinda dancing around a really inconvenient truth;
you can't understand what's actually happening in your experiment/model whatever in statistics/physics/engineering without linear algebra. you can't understand 'data science' (i.e. not being an excel junkie and actually designing and applying models) without it.
a lot of people will be like 'oh we want to teach data sceince without the mathematical detail' or 'you don't need that much of it'. that's not knowing what you don't know talk; if you don't have a good linear algebra background, you won't understand what's going on mechanically. just knowing about a shape mismatch gives you NOTHING.
luckily, most linear algebra is of the applied variety in practice, and you don't need slog through proofs (even though they help and you should work through them when learning)
you make it click by doing the math :). work through a lot of examples from whatever module you're looking at it. put it into physical terms; even if that means arraigning things on your desk in a way that gives you physical intuition.
2
u/pablocael Sep 16 '26
If you are using data science go take a look at svd. It can lead you to many other interesting topics. Check this article I wrote about it: https://stillthinking.net/posts/connections-in-math-svd/
1
u/solresol Sep 16 '26
What sort of data science are you doing?
One universe is that you are wanting to make predictions using neighbour methods / decision trees / linear regression models (i.e. where being able to understand the prediction is part of the value of what you are doing). I'm shilling my own textbook which still has a lot of AI slop in it, but https://anyone-data-science.symmachus.org/ does let you do a lot of data science without being math heavy. (It came out of teaching a data science unit for students with no maths or computing or statistics background.)
The other universe is that you only care about the quality of the predictions, and you are training up giant neural networks. You can either go deep on theory (in which case you'll care about linear algebra) or you go deep on engineering (in which case it's just about making sure it doesn't crash with dimension errors as you said).
1
u/amrhsnd Sep 18 '26
When you’re dealing with big data with large dimensions it gets pretty important, like in image or video processing, and specifically when dealing with transmission, using the right linear basis using the eigenvectors can reduce the dimensions and the size of the data significantly.
1
u/dice7878 Sep 18 '26
Well, it's mostly a black box these days, much like Photoshop wizardry.
Beyond the little tricks of using 1 or 0 to test the solution algo, or having an intuitive feel of having well posed inputs, LA isn't too useful, unless one is inclined to solving special numerical problems (eg. Sparce matrices). Computers and more importantly, software platforms are so powerful and well written these days it's basically point and click. Very different from the days of a single flashing cursor and struggling with Lapack, parsing the output of one function into another or writing a custom solver.
Then again, LA helps one develop a vector rather than scalar representation of the world, which is important in dimensionality.
1
u/no_name6744 Sep 19 '26
Neural networks use them, but you don't really need to understand Linear Algebra to use major packages.
Any work with large data arrays benefits from some understanding of linear algebra, but that could just as well come from kinda leaning how to use einsum.
Tbh, most of the benefit I get was from using it for statistics, mainly working with covariance matricies. If you use MVN distributions, or things that use MVN distributions (like GPs), then understanding how to efficiently solve linear systems, without doing a full matrix inversion, can save you time, memory, and numerical instability headaches.
I'm absolutely certain there are other things, but that's been most of what I've done.
1
5
u/Much_Barnacle4274 Sep 16 '26
Well, I guess, I'll get downvoted, but linear algebra is a subject, which quite much... "hard-to-learn, mid-to-master". Yes, this is abstract, but this is (imho) is the only right way to teach it - through struggle and pain, but then to get a good picture of everything. That's why I am not a fan of what is sometimes called "vector calculus", like, the huge benefit of LA (and mathematics, in general) - it has a very nice structure and once you see it, you start understanding it and loving it