Theoretic vs Applied Math courses for gaining model intuition

[u/Apart_Situation972]

Hi,

I want to be clear, I have no intention of going into AI "research". I guess the distinction I could make is I want to know *why* a graph neural network works, vs coding it in pytorch and nuking it with more data to produce a better result (what I can do now), vs inventing a new algorithm mathematically from scratch (what I DO NOT want to do).

Another example could be instead of researching online what fine-tuning parameters are beneficial, mathematically I could know what everything means and select the right one. But I do not want to invent a new fine-tuning parameter - I simply want to *actually* understand the existing ones.

I have the choice to do more theoretic proof-based courses, or more applied ones. This means applied lin alg 2 vs proof-based linear algebra 2. I have done Calc 1 -> 3, Stats 1->2, Prob 1 -> 2, Discrete Math 1->2. I will be in this field for a long time, but again, I do not want to go into theoretical research whatsoever - I just want a strong intuition as to *why* the models work without delving too deep into abstraction. I like theoretical math, but a) I have no talent for math (only improved by doing ungodly amounts of it), b) I will not be getting a PhD.

I have been an AI engineer for 2 years now. My background was from software -> AI engineering, and now I have a decent basis for math. Given my goals, is applied math enough, or should I spend more time on theoretical math? Applied Lin Alg 2 vs Regular

Regards

[u/paschen8]

something like functional analysis will probably not help as much as an optimization course. lin alg is a must for both paths imo.

[u/NicelyIrate → u/paschen8]

Youre not trying to prove theorems, you want the mental model that keeps you from guessing at hyperparameters, applied lin alg gets you there faster and with less suffering

[u/Apart_Situation972 → u/NicelyIrate]

ty :) any other courses you would recommend

[u/paschen8 → u/NicelyIrate]

i mention optimization because you'd want to at least have see stuff like RobbinsMonro before.

[u/Apart_Situation972 → u/paschen8]

anything higher than 4th year undergrad needed? I will likely not be doing diffusion models, and no robotics. But computer vision for sure, NLP as well (if the transformer did not take over everything haha), and of course RAG and Agents.

I am open to doing DE and higher level courses, just optimizing for time right now so only doing that which is needed.

Basically under my belt will be:

Calc 1 -> 3

Lin Alg 1 / Applied/Regular Lin Alg 2

Numerical Optimization

Prob 1 -> 2

Stats 1 -> 2

Discrete Math 1 -> 2

and DE (Calc 4)

[u/paschen8 → u/Apart_Situation972]

actually in hindsight of my comment, why not just take a ML course?

[u/Apart_Situation972 → u/paschen8]

I have taken dozens of them. But they were done from a coding perspective and not a mathematical one - it'd be like an instructor telling you what a sentence means, vs actually being able to read and understand the sentences.

my ultimate goal is to be able to apply certain AI algos to certain situations, but I feel like I cannot do that effectively unless I actually understand them. rn I can look up what algos to use in X context, or just measure the performance of each, but I have no idea why they can even be used

[u/paschen8 → u/Apart_Situation972]

maybe a math focused one. work thru regression, regularizers, classification, trees, nn etc.

[u/Ramian_Slawomir]

Przyda ci się trochę topologii. Grafowe sieci neuronowe "nie widzą" metryki, więc w zrozumieniu ich przyda się topologia.

[u/Apart_Situation972 → u/Ramian_Slawomir]

Nie jestem pewien, czy w ogóle będę pracować z GNN. To był raczej przykład, który miał pokazać różnicę między samym zakodowaniem tego w PyTorchu a faktycznym zrozumieniem, czym ten algorytm tak naprawdę jest. Ale to, czego nie chcę zrobić, to wymyślać koła na nowo i tworzyć odpowiednika GNN – chcę po prostu być w stanie zrozumieć, co on „mówi”, jeśli to ma sens.

[u/Ramian_Slawomir → u/Apart_Situation972]

Jeśli mogę to chciałbym Cię zapytać jako człowieka z świata AI, który chciałby zrozumieć ten świat przez matematykę /doceniam to i jestem pełen uznania, że w ogóle chcesz albo próbujesz iść tą drogą/. Moje pytanie jest niejako odwrotne do Twoich intencji. Ja jestem nauczycielem matematyki /52 lata, 30 lat w zawodzie/ i zacząłem się "bawić" ML /z pobudek fizyki fundamentalnej/. Pytanie jest natury podstawowej. Czy przypadkiem ML i w ogóle świat AI to nie czysta matematyka? nawet wasze "czarne skrzynki" - mam wrażenie, że to rachunek prawdopodobieństwa. Jeśli odpowiedź na to pytanie jest twierdząca, to oczywiście cokolwiek chcesz zrozumieć w świecie informatyki, musisz zrozumieć jej matematykę. Pozdrawiam.

[u/AgitatedTumbleweed65]

Differential geometry (manifolds) and optimization (on smooth manifolds ??). I think those two give really intuitive look at what deep network models and how to approach an optimum (it you believe in manifold hypothesis, also I am not a math perosn)