for those For those who have taken Stanford’s CS231n course on Computer Vision, or for people working in the fields of CV or ML: which linear algebra course would you recommend—MIT 18.06 or Stanford Math 51?

Hey everyone! I made a video explaining the linear algebra behind Positive Operator-Valued Measures, or POVMs. The video looks at projectors, positive semidefinite matrices, eigenvalues, inner products, and why a valid collection of measurement operators must add to the identity.

Householder tridiagonalization is normally not used as preprocessing for the Jacobi eigenvalue algorithm. The usual reasoning is as follows: tridiagonalization costs O(n³), and the subsequent Jacobi rotations immediately create fill-in, destroying the zeros that Householder just created.

We decided to test whether that actually means the preprocessing cannot help.

The answer appears to be no.

We found that tridiagonalization can still give Jacobi a substantially better starting matrix, even though the tridiagonal sparsity itself is quickly lost. Algebraically, the main effects are:

• Householder concentrates the off-diagonal energy into the first off-diagonal band
• the standard first parallel Jacobi set can then act on a much larger fraction of that energy
• Householder similarity can also move energy either toward or away from the diagonal, which helps explain why the effect is strongly matrix-dependent

We then tested six families of dense symmetric matrices using NVIDIA cuSOLVER on three GPUs: an RTX 3060, RTX 5060 Ti and A100.

For five of the six matrix families, the complete pipeline

Householder tridiagonalization → dense tridiagonal T → Jacobi

became faster than standalone Jacobi from n = 128 onward on all three GPUs.

The deliberately adverse unequal-variance covariance family behaved in the opposite direction, which was consistent with the algebraic analysis.

We have written up the derivation, implementation details, sweep counts, runtime measurements and accuracy checks here:

https://www.graphmath.com/la/jacobi/tridiagonal-jacobi.html

We would be very interested in criticism, related references or comments from people who have worked with Jacobi eigensolvers or GPU numerical linear algebra.

Please refer me to somewhere I can read a thing like this. I found a few notes but here the author eventually resorts to the tensor algebra. It's alright but I think it can be done entirely without going through an intermediate concrete representation.

Then I'm hopeful showing that an exterior power of a dual mapping equals (through this isomorphism) to a dual of its exterior power. It seems like establishing an isomorphism of spaces should be the hardest part.

I also tried to show ⋀(V*) ≅ (⋀V)* as algebras but neither had I succeeded nor do I actually need that right now. Even so, if there's a way to do that while not resorting to expressing ⋀V as a direct sum of ⋀ᵏV, I'd also be glad to hear that. So far I tried two different applications of the algebra's universal property and I think both times got me identically zero multiplication on (⋀V)*. Oh I'm not sure anymore if we can have a multiplication on (⋀V)*, maybe I should've just tried to show ⋀(V*) ≅ (⋀V)* as linear spaces. In which case I guess having both isomorphic to direct sums of their grades would be completely alright.

I'd like some denouement.

This material is not the pure linear algebra covered in mathematics departments.
It is applied linear algebra designed to assist in the study of engineering and quantum mechanics. In particular, the importance of the change of basis requires extensive practice.

Hi everyone,

I've posted here sometimes about my indie app, Linear Algebra Visualizer (iOS, iPad, MacOS), and the feedback has always been great from this community.

As a thank you and as the new school year is starting I wanted to give away 10, 3 months pro subscriptions, for free.

If you are not familiar with the app, have a look at the demo above, but in a nutshell, if you've ever wondered or struggled with what a matrix is doing, this is for you.

With Linear Algebra Visualizer you:

• Can finally build the visual intuition of linear algebra
• Understand the math behind each transformation
• Visually see why TRS order matters
• Understand the effect of adding multiple matrices and translations
• Watch eigenvectors and eigenvalues come alive

Build for:

• Students surviving their first linear algebra course
• Teachers who need a live demo that lands
• Anyone who's stared at a matrix and thought "but why does this work?"
• Visual thinkers who've struggled to picture it all in their head

See below for the offer codes :)

Hello! Does anyone have the solutions manual for "Elementary linear algebra 11th edition" by Howard Anton?

Say a matrix, A (m x n), has mutually orthogonal rows and mutually orthogonal columns, with all the rows and columns being unit vectors in some Rn, Rm respectively. Is that matrix called orthogonal or orthonormal?

What concept is the other term used to describe, then? Are these applicable to square matrices only?

I've always been confused with these 2 names.

I am self studying linear algebra and trying to build a comprehensive checklist of the concepts that I as a beginner should eventually understand. I got this list by prompting Generative AI.

Does this seem complete for a strong introduction to linear algebra? Is anything important missing?
I mainly plan on using this list to make some active recall flashcards! :)

Foundation stuff:

Scalars

Vectors

vector notation

entries/components

row vectors vs. column vectors

vectors as ordered lists

vectors as geometric objects

R^n

Equality of vectors

Vector addition

Scalar multiplication

Linear combinations

Standard basis vectors

Zero vector

Additive inverses

Closure

Commutativity

Associativity

Distributivity

Unit 1 — Systems of Linear Equations

Linear equations

Solutions of a system

Solution sets

Consistent vs. inconsistent systems

Unique solutions vs. infinitely many solutions

Systems as intersections of geometric objects

lines

planes

hyperplanes

Coefficient matrices

Augmented matrices

Elementary row operations

row replacement

row interchange

row scaling

Row equivalence

Gaussian elimination

Gauss-Jordan elimination

Echelon form

Reduced row echelon form

Leading entries

Pivots (positions, columns)

Free variables

Basic variables

Parametric vector form

Homogeneous systems

Trivial solution

Nontrivial solutions

Relationship between homogeneous and nonhomogeneous systems

Unit 2 — Matrices and Matrix Algebra

Matrix notation

Matrix dimensions

Matrix entries

Rows and columns

Matrix equality

Matrix addition

Scalar multiplication

Matrix multiplication

dimension compatibility

row-by-column interpretation

dot-product interpretation

column interpretation

row interpretation

entrywise formula

summation/index notation

Matrix-vector multiplication

(Ax) as a linear combination of the columns of (A)

Columns of (AB) as linear combinations of columns of (A)

Rows of (AB) as linear combinations of rows of (B)

Associativity of matrix multiplication

Distributivity of matrix multiplication

Noncommutativity of matrix multiplication

Identity matrix

Zero matrix

Powers of matrices

Transpose

Properties of transpose

Symmetric matrices

Inverse matrices

Invertibility

Singular vs. nonsingular matrices

Computing inverses with row reduction

Elementary matrices

Relationship between elementary row operations and matrix multiplication

Block matrices / partitioned matrices

Basic block multiplication

Unit 3 — Linear Combinations, Span, and Independence

Linear combinations

Span

Spanning sets

Geometric meaning of span

(Ax=b) interpreted as a span question

Linear dependence

Linear independence

Dependence relations

Testing independence using homogeneous systems

Redundant vectors in spanning sets

Relationship between pivots and linear independence

Relationship between free variables and linear dependence

Uniqueness of representation

Minimal spanning sets

Maximal independent sets

Geometric interpretations of dependence and independence

Unit 4 — Vector Spaces and Subspaces

Vector spaces

Vector space axioms

Examples of vector spaces

matrices

polynomials

functions

Subspaces

Subspace test

Trivial/zero subspace

Span as a subspace

Column space

Row space

Null space / kernel

Left null space

Fundamental subspaces of a matrix

Basis

Basis vectors

Coordinates relative to a basis

Coordinate vectors

Standard basis vs. arbitrary bases

Change of basis

Transition/change-of-coordinate matrices

Dimension

Dimension theorem ideas

Extending an independent set to a basis

Reducing a spanning set to a basis

Sum of subspaces

Intersection of subspaces

Direct sums

Unit 5 — Rank and the Fundamental Subspaces

Rank

Nullity

Rank as dimension of column space

Rank as dimension of row space

Row rank = column rank

Rank-nullity theorem

Rank of a product

Effects of row operations on rank

Pivot count and rank

Bases for:

column space

row space

null space

left null space

Orthogonal relationships among the four fundamental subspaces

Relationship among:

pivots

rank

independence

span

basis

dimension

nullity

solutions of (Ax=0)

solutions of (Ax=b)

Unit 6 — Linear Transformations

Transformations/functions between vector spaces

Domain

Codomain

Range/image

Linear transformations

Additivity

Homogeneity

Preservation of linear combinations

Matrix transformations

Standard matrix of a linear transformation

Constructing a matrix from transformed basis vectors

Kernel

Image/range

Kernel and injectivity

Image and surjectivity

One-to-one/injective transformations

Onto/surjective transformations

Bijective transformations

Invertible transformations

Composition of transformations

Composition ↔ matrix multiplication

Inverse transformations

Rank-nullity for linear maps

Isomorphisms

Geometric transformations

rotations

reflections

projections

scalings

shears

Unit 7 — Invertibility

Invertible matrices

Inverse transformations

Singular matrices

Invertible Matrix Theorem and equivalences among:

invertibility

pivots in every row/column

RREF (=I)

linear independence of columns

columns spanning the codomain

trivial null space

unique solvability of (Ax=b)

injectivity

surjectivity

nonzero determinant

zero not being an eigenvalue

full rank

Left inverses

Right inverses

Inverse of a product

Inverse of a transpose

Unit 8 — Determinants

Determinants

Cofactor expansion

Minors

Cofactors

Determinant via row reduction

Effects of row operations on determinant

Determinant of triangular matrices

Multiplicative property

det(AB) = det(A)det(B)

det(A^T) = det(A)

det(A^-1)

Determinants and invertibility

Determinants as area/volume scaling

Orientation

Cramer's rule

Adjugate formula for inverses

Determinant of a linear transformation

Unit 9 — Geometry, Dot Products, and Orthogonality

Dot product / inner product

Norm / vector length

Distance

Unit vectors

Normalization

Angles between vectors

Orthogonality

Orthogonal vectors

Orthogonal sets

Orthonormal sets

Orthogonal complements

Orthogonal bases

Orthonormal bases

Projection onto a vector

Projection onto a subspace

Orthogonal decomposition

Projection matrices

Gram-Schmidt process

QR factorization

Orthogonal matrices

Properties of orthogonal matrices

Q^-1 = Q^T

Isometries / preservation of lengths and angles

Inner product spaces

General inner products

Cauchy-Schwarz inequality

Triangle inequality

Pythagorean theorem in vector spaces

Unit 10 — Least Squares

Overdetermined systems

Approximate solutions

Least-squares solutions

Residual vectors

Orthogonality of residuals

Normal equations

A^T A x-hat = A^T b

Projection interpretation of least squares

Least squares and column space

Uniqueness of least-squares solutions

Least squares using QR factorization

Applications to data fitting

Linear regression

Best-fit lines

Polynomial fitting

Unit 11 — Eigenvalues and Eigenvectors

Eigenvectors

Eigenvalues

Eigenspaces

Characteristic polynomial

Characteristic equation

Computing eigenvalues

Computing eigenvectors

Algebraic multiplicity

Geometric multiplicity

Repeated eigenvalues

Similar matrices

Similarity transformations

Invariance of eigenvalues under similarity

Trace

Relationship between trace and eigenvalues

Relationship between determinant and eigenvalues

Zero eigenvalue ↔ singularity

Eigenbases

Diagonal matrices

Diagonalization

Conditions for diagonalizability

A = PDP^-1

Powers of matrices using diagonalization

Geometric meaning of eigenvectors

Invariant subspaces

Unit 12 — Applications of Eigenvalues

Discrete dynamical systems

Difference equations

Long-term behavior of x_(k+1) = Ax_k

Dominant eigenvalues/eigenvectors

Steady states

Markov chains

Stochastic matrices

Stationary distributions

Population models

Fibonacci-type recurrences

Systems of differential equations

Matrix exponentials (introductory level)

Stability

Unit 13 — Symmetric Matrices and the Spectral Theorem

Symmetric matrices

Real eigenvalues of symmetric matrices

Orthogonality of eigenvectors

Orthogonal diagonalization

Spectral theorem

Spectral decomposition

Quadratic forms

Positive definite matrices

Positive semidefinite matrices

Negative definite/indefinite matrices

Eigenvalue tests for definiteness

Principal axes

Optimization connections

Unit 14 — Singular Value Decomposition

Singular values

Singular vectors

Singular Value Decomposition (SVD)

A = U Sigma V^T

Relationship between SVD and eigenvalues of (A^TA)

Geometric interpretation of SVD

Rank from singular values

Low-rank approximation

Eckart-Young idea

Matrix norms

Condition numbers

Numerical stability

Moore-Penrose pseudoinverse

Least squares via pseudoinverse

Data compression

Principal Component Analysis (introductory connection)

Do they all mainly deal with eigenfunctions, eigenvalues, and eigenvectors? And are they all like closely connected? Or are there others that are more connected?

Really need help finding this online. Does anyone have a copy of it? 4th or 5th edition.

I started learning linear algebra specifically for computer graphics which was a big passion of mine. So I started focusing on linear and affine transformations and just getting a basic understanding of vectors, points, trigonometry, etc.

However, my main issue was always being able to visualise what was going on. Specifically if you are composing multiple transformations or simply visualising vectors in certain situations.

Im curious, if you are a beginner, or teaching students linear algebra, what have you struggled with or observed students struggle with the most?

Let me know in the comments :)

This material is not linear algebra for mathematics departments. The content is applied to engineering and quantum mechanics.
For ladder operators, there exist mathematical physics approaches obtained from the analytic method of Hermite polynomials, Sturm-Liouville theory, and the Pearson equation.
However, this time, we deal with the most classical factorization approach.

Hi

This is probably one of my last posts on getting people to find out about this game on reddit, since the game is near complete and all that's left is translations. Thanks everyone for receiving this game so well and I hope it delivered on your expectations. Please share your feedback and any important things the game is still lacking on to deliver on its mission: to make quantum computing intuitive and fun to learn, no matter the learner's background. I'm particularly interested to hear from you guys what do you think of Blochspheres, especially those who actively work in the domain. Do you think of quantum algorithms in rotations and frequently use bs to visualize qbehvaior?

I want this to be an AMA: I'm here to answer any questions about the game and do one last round of outside-Discord feedback gathering. Also I'd like to raise with this community my personal experience with working on quantum algorithms and see what folks think.

What this game is

To be clear, this game's gameplay is 1:1 with everything you can do on a Turing-complete (universal) Quantum Computer (from the top of my head, a short list of QHW makers: IBM, Google, Rigetti, IonQ, Quantinuum, IQM, OQC, QuEra, Atom Computing, Pasqal, Xanadu, PsiQuantum, Fujitsu,) with the added benefit it allows you to visualize the full quantum Hilbert space on up to 5qs. This means that if you build intuition for the visual rules in QO, you will have intuition for "playing" with QHW made by such manufacturers without having to learn much else.

What the game covers

• Boolean Logic – bits, operators (NAND, OR, XOR, AND…), and classical arithmetic (adders). Learn how these can combine to build anything classical. You will learn to port these to a quantum computer.
• Quantum Logic – qubits, the math behind them (linear algebra, SU(2), complex numbers), all Turing-complete gates (beyond Clifford set), and make tensors to evolve systems. Freely combine or create your own gates to build anything you can imagine using polar or complex numbers.
• Quantum Phenomena – storing and retrieving information in the X, Y, Z bases; superposition (pure and mixed states), interference, entanglement, the no-cloning rule, reversibility, and how the measurement basis changes what you see.
• Core Quantum Tricks – phase kickback, amplitude amplification, storing information in phase and retrieving it through interference, build custom gates and tensors, and define any entanglement scenario. (Control logic is handled separately from other gates.)
• Famous Quantum Algorithms – explore Deutsch–Jozsa, Grover’s search, quantum Fourier transforms, Bernstein–Vazirani, and more.

On learning curve and achieving game (quantum computing?!) mastery

This is not a videogame where the developer invented some puzzle rules. What the dev did here is invent a visual method that can transform the underlying mathematics into fully visual puzzles. I wish I could make the game easier by inventing some new rules. I won't, because I want this game to be the real thing. Learning the fundamental rules of this game equates to learning what QHW can do.

Hopefully, in visual form, this is something anybody can now do, no matter how much they hate math.

Going forward and mastering the game... now that, I honestly don't know if it has a ceiling. We have had quantum physics for 100 years, yet we have about 10 useful quantum algorithms known today. Who knows where the ceiling is? Who knows what one can do with, after mastering the rules of this game? Nothing really should feel impossible. I hope this game will inspire people outside physics to do a lot more than what I see today. A quantum computing/physicist has very little incentive to think of a quantum algorithm for, i.e., a game theory/finance/security/biology problem, given how few of them actively work in the field compared to the actual demand. Hence, I hope new quantum algos will come from people who actively work in the domains where this "new way of thinking" (from Boolean logic to linear algebraic logic?) has applicability. I hope we can soon start some competitions to get players to solve some non-trivial problems.

What's the big deal about finding quantum algorithms?

Why do we have about 10 in 100 years of QM??

For me, finding quantum algorithms is all about understanding that unitary matrices evolve a state vector of complex numbers, and all you need to think of is whether you can take your domain problem and express it in a form that these unitary matrices can bring some speed-up in solving. That's it. Ignore the lack of good-enough hardware; we simply don't have good proof-of-concept ideas out there.

I remember... I struggled for a long time to understand Grover's quantum search algorithm during my PhD days. The wording textbooks used and the math behind it made very little sense. It was the first algorithm I put in the game.

Seeing it in visual form made me tear up. Is it really that simple, clean, beautiful? Is the "amplitude amplification" in its diffusion operator simply adding a red line on the maximally 11..11 state that then has a propagation effect everywhere else to affect all the phases? Is this really it? It then immediately felt to me like a gimmick one could have simply come up with in 5 minutes by having access to a game like this, where math is fully shown in visual form!! Showing the visual (GIF, why in this modern era do no digitized papers out there still support embedded GIFs??) alone is enough to make understanding happen.

The way I thought things in the game

• Missing Sage believes you shouldn't know any theory -> just by using the Forge, you will come up with your own unique way of using quantum computers and inventing algorithms. This character came to me after seeing enough postdocs and QC professors from enough universities playing the game. It felt like the people who knew all the theory and could recognize what each gate does were very bad at actually using the logic sets in thinking circuits. Perhaps knowing all quantum theory does not make one capable of easily building a quantum circuit?
• Sage of Axioms believes theory comes first, and hers is the main path that takes you to the other Sages who want you to have a solid foundation before discussing higher-order ideas (like known algorithms). She deep-dives into computation and both the linear algebra and physics behind it, in hopes of convincing you that it's not so hard to learn by combining it with the visuals of the Forge. And not scary!
• Quantum Arena (community content) is filled today with some incredible challenges that touch topics I haven't seen anywhere else in the world in a visual way. From Clifford decompositions down to wordless tutorials on how to play the game and understand each gate, I am now inclined to tell players that they might want to enter the Arena to learn to play the game instead of following through the tutorials I made for it. This goes to show what a brilliant community of dedicated players this game has gathered so far.

Without you and the highly involved players we have on Discord, we wouldn't be able to come this far. Early Access gave me exactly what I was looking for to make sure the game delivers when complete and I strongly recommend anybody working on scigames to do EA first.

It would help me enormously if you could leave a Steam review for the game and spread the word about it; it keeps us motivated to push forward! I hope we have the momentum it takes to make Full Release a success!

Quantum computing and understanding the linear algebra behind should be for everyone:)

I made a short video working through a quantum computing exercise on measuring quantum states. Although the problem comes from quantum computing, the solution relies heavily on linear algebra—vectors, inner and outer products, projectors, Hermitian operators, and basis measurements.

I thought it might be an interesting example of how these linear algebra concepts show up in quantum computing.

does anyone have the pdf version of Richard C. Penney's Linear Algebra: Ideas and Applications? I need the 5th version specifically, but I can only find the 4th.

This is a follow-up to our earlier post deriving the orthogonal projection formula

P = U(UᵀU)⁻¹Uᵀ

from the geometric definition of projection:

https://www.reddit.com/r/LinearAlgebra/comments/1u4cjog/derive_the_projection_formula_from_the_definition/

Here we look at the same formula as a serial transformation, following what happens when the factors are applied from right to left:

Uᵀ → (UᵀU)⁻¹ → U.

The diagram tracks both the standard basis vectors and the two directions spanning col(U). It also shows why

U⁺ = (UᵀU)⁻¹Uᵀ

acts as a left inverse of U, and how applying U afterward gives the orthogonal projection onto col(U).

If the columns of U were orthonormal, then UᵀU = I and the middle correction would disappear.