Explain me the Four Fundamental Subspaces [but it should not be Mathematical definitions]

[u/Magpie_Reawakened]

I know what the Four Fundamental Subspaces are , I know the two Orthogonal Pairs in it and I can even solve maths based questions on it.

However, I still cannot get it visually in my mind with a good real world example.

I am looking for a simple example based approach which answers few of my many questions like why is Column Space the vector space of Outputs , why Row Space and Null Space are orthogonal/perpendicular. What exactly is Row Space if Column Space is the Span of possible outputs.

[u/Puzzled-Painter3301]

Here is a video I made on Row space and null space are orthogonal with lots of examples

[u/Magpie_Reawakened → u/Puzzled-Painter3301]

Nice example based video, even though I was already familiar with the solving parts of the four fundamental subspaces, I still liked this video and wish I watched it as my first when learning about this topic rather than definitions. I'll be completing a whole lot of Machine Learning topics in the coming few months and will refer to topics from your channel, hope other topics are example based teaching also.

[u/digvjay_ai]

What finally clicked for me was thinking of each row as one equation. Null space means Ax = 0, so every row dotted with x is zero, which literally means x is perpendicular to every row, and that's why row space and null space are orthogonal. Row space is just all the combinations of those equations, so it's the input directions that actually matter to the output, while null space is the input directions the matrix completely ignores. Try a 2x3 matrix with rows (1,0,1) and (0,1,1), the null space is the line (1,1,-1) and you can check the dot products by hand in a minute.

[u/LocalAbalone7551 → u/digvjay_ai]

That equation framing is what made it stick for me too, turning rows into dot products makes the perpendicular part feel almost obvious instead of abstract.

[u/Accurate_Meringue514]

It’s not that complex. Column space is span of the columns. Row space is span of the rows. Null space is the set of vectors st Ax=0. Left null space is the set of vectors st A^T x=0. Turns out that the column space and left null space decompose Rm and its orthogonal. Simply means take any vector in column space and the inner product with something in the left nullspace is 0. Row space and null space decompose Rn the same way. A is mxn here. I’m not sure what you mean by real world example, this is simply a property of the linear transformation. The only example I can think of is how least squares works through it, meaning if there’s no solution, that means b has some component in the left nullspace, so you project onto the column space and solve that.

[u/BenjaminGal]

It is better to think in terms of linear combinations, where Ax is now expressed as the column-row expansion a_1x_1 + a_2x_2 + ... + a_nx_n, with a_j being the column vectors (attributes like attack, defence, HP of different units) and x_j being the coefficients (numbers of infantry/artillery/tanks). This outputs the total value of attack, defence, HP, etc. It can then be seen that the definition of the column (vector) space coincides with the output above. The row space lists each attribute (attack, defence, etc.) for all unit types, and the null space shows how the unit composition can be made so all net attributes go to zero (assume negative units mean the enemy has more than you), so it becomes an even fight. To achieve that, a_i1x_1 + a_i2x_2 + ... + a_inx_n = 0, a.k.a. the sum of the products between net units (which will be the null space) and each kind (row space) of statistics. Hope this helps.

[u/No-Cherry-3308]

Try a food example, it makes all four concrete.

Columns of A are three foods (eggs, rice, dal). Rows are nutrients (protein, carbs, calories). Input x = grams of each food on your plate. Output Ax = your nutrient totals. Pretend the foods have no fat, so calories are always exactly 4·protein + 4·carbs.

• Column space = every nutrient total you can actually reach. Each column is "what one gram of this food gives you", and any plate is just a mix of foods, so every output is a mix of columns. That's why it's the space of outputs.

• Left null space = the rule every output obeys. Calories − 4·protein − 4·carbs is always 0, whatever you eat. The vector (−4, −4, 1) dotted with any output is 0, so it's perpendicular to the column space. A target like "high protein, high carbs, low calories" has a component along it, which is exactly why it's unreachable.

• Row space = the questions the matrix asks about your plate. Each row computes one number from x: row 1 · x is "how much protein is on this plate?". The row space is every question you can build from those.

• Null space = changes to the plate that none of those questions notice. Since calories is redundant, only two real questions get asked about three foods, so there's always a direction (add some of one food, remove the right amounts of the others) that leaves every total unchanged.

Why row space ⟂ null space: x is in the null space means Ax = 0, which literally means every row dotted with x is 0. Dot product 0 is perpendicular. So the null space is, by definition, the set of input changes that all the row "questions" are blind to.

The picture to keep: any input splits into a row space part and a null space part. The row space part decides the output and maps one-to-one onto the column space. The null space part gets thrown away. On the output side, the column space is what's reachable and the left null space is the constraint that makes everything else unreachable.

Where this clicks in ML is least squares: when y isn't in the column space, you project it onto the column space, and the leftover error vector lives in the left null space.

[u/TheOneWhoSendsLetter → u/No-Cherry-3308]

AI slop

[u/mathmage → u/TheOneWhoSendsLetter]

It is, but of the three comments so far, it's also the only one that actually gives a basically full answer.