Matrices between dimensions
What you’ll learn
Section titled “What you’ll learn”This is lesson 8 of Track 4 (Visual Math: Linear Algebra). Every matrix so far has been square, quietly assuming input and output have the same number of dimensions. This lesson drops that assumption. By the end you will be able to take any rectangular matrix and classify it completely: its input dimension (the number of columns), its output dimension (the number of rows), its rank, and what it does, whether it embeds a small space intact inside a bigger one or projects a big space down into a smaller one. You will see why a projection always discards a direction (it always has a null space), and why the column space, rank, and null-space tools from the previous lesson carry over unchanged. The source curriculum is 3Blue1Brown’s Essence of Linear Algebra by Grant Sanderson, freely available at 3blue1brown.com.
Where this fits
Section titled “Where this fits”This is lesson 8 of 15, the third lesson of Phase 2 (geometry of operations). It generalizes everything from Inverses, column space, and null space: rank, column space, and null space all work the same way, the only new thing being that input and output dimensions can now differ. That difference makes the rank-nullity balance more interesting, since a full-rank projection still has a null space. The next lesson, Dot products and projection, looks at the most extreme rectangular case: a one-row matrix that turns a vector into a single number, which turns out to be the dot product in disguise.
Before you start
Section titled “Before you start”Prerequisites: the previous lesson, Inverses, column space, and null space. You need to be comfortable with rank (the dimension of the column space), the column space as the span of the columns, and the null space as the inputs sent to zero, because this lesson applies all three to matrices that change the dimension. No new computation is introduced. The practice is pen and paper.
About the math
Section titled “About the math”The arithmetic is the same matrix-vector product you know, now producing an output with a different number of entries than the input. You will apply an embedding and a projection by hand, and classify a few matrices by reading their shape and checking whether their columns are independent. The work is interpretation (which space is which, what the rank and null space say) rather than heavy calculation.
By the end, you’ll be able to
Section titled “By the end, you’ll be able to”- Read an m by n matrix’s shape to find the input dimension (columns) and output dimension (rows)
- Distinguish an embedding (small space into big) from a projection (big space into small) by the matrix shape
- Locate the column space in the output and the null space in the input, and compute the rank of a rectangular matrix
- Explain why a projection always has a nontrivial null space, even at full rank
- Classify any rectangular matrix by input dimension, output dimension, rank, and direction of mapping
Time and difficulty
Section titled “Time and difficulty”- Read time: about 10 minutes
- Practice time: about 15 minutes (a classify-the-matrix exercise, an apply-across-dimensions drill, and flashcards)
- Difficulty: standard (a Phase 2 lesson; the tools are all reused, the new part is reading the change of dimension)