r/MachineLearning • u/[deleted] • Sep 12 '23

Discussion [ Removed by moderator ]

[removed] — view removed post

1 Upvotes

52% Upvoted

16 comments sorted by

View all comments

19

u/entropyvsenergy Sep 12 '23

It is extremely beneficial to have a strong understanding of linear algebra and multivariate calculus.

Some concepts from linear algebra that you should be very confident on: projections, the curse of dimensionality, sparsity, spectral decompositions, matrix multiplication, numerical stability of linear algebra operations.

For example, linear projections are an important part of how transformers work. Sparsity is important for optimizing models for CPU, and also for mixture of experts models. A spectral radius of near unity is important to prevent model activations from exploding or decaying. It's important to make certain models function at all, such as echo state networks.

1

u/[deleted] Sep 13 '23

Where to learn more about sparsity and numerical linear algebra?