Lesson: Undoing a transformation, and when you cannot

Last lesson ended on a sharp claim: when a transformation has determinant zero, it collapses space and information is lost for good. This lesson takes that claim apart and makes it precise. What does it mean to undo a transformation, when can you, and when you cannot, what exactly got destroyed?

Three ideas do the work: the inverse (the undo button), the column space (everything the transformation can reach), and the null space (everything it crushes to nothing). Together they answer one practical question that runs through all of applied math: given a transformation, can you reverse it, and if not, why not?

Systems of equations are secretly transformations

Start with something that looks unrelated: a system of linear equations.

2x + 5y = 3
4x + 3y = 7

Stack the coefficients into a matrix and the unknowns into a vector, and the whole system becomes a single equation:

[ 2  5 ] [ x ]   [ 3 ]
[ 4  3 ] [ y ] = [ 7 ]

That is just M times the unknown equals the target, where M is the coefficient matrix, the unknown is the input vector, and the target is the vector 3, 7. Solving the system means finding the input vector that the transformation M sends to the point 3, 7. Every linear system is the question “which vector lands here after this transformation,” and that reframing is what makes the rest of the lesson useful.

The inverse is the undo button

If M moves space one way, the inverse, written M-inverse, is the transformation that moves it back. Apply M to a vector, then apply M-inverse, and you are exactly where you started. Doing both in sequence changes nothing, which we write as

M^-1 · M = M · M^-1 = I,   where   I = [ 1  0 ]
                                       [ 0  1 ]

I is the identity matrix: its columns are i-hat and j-hat unchanged, so it leaves every vector alone. The inverse is defined by this single property: composed with M, it gives back the do-nothing transformation.

Once you have M-inverse, the system solves itself. From M times the unknown equals the target, apply M-inverse to both sides, giving the unknown equals M-inverse times the target. The solution is the target run backward through the inverse. This lesson is about what M-inverse is and when it exists, not the mechanical recipe for computing its entries; software does that, and the meaning matters more.

When the inverse exists: nonzero determinant

Here is the bridge from last lesson. The inverse exists exactly when the determinant of M is nonzero.

The reason is the collapse story. If the determinant of M is nonzero, the transformation only rearranged space without flattening it, so every output came from exactly one input, and you can run it backward unambiguously. If the determinant of M is zero, the transformation crushed space onto a line or a point, many different inputs landed on the same output, and there is no way to know which one to go back to. The information is genuinely gone, so no undo transformation can exist. Zero determinant, no inverse. That is the whole condition.

If a transformation keeps the grid spread across the full plane, it is invertible: every output traces back to one input. If it crushes the plane onto a line, the transformation is collapsed: many different inputs land on the same output, and there is no way to undo it.

Column space: everything the transformation can reach

Now the two ideas that describe a collapse precisely. The first is the column space: the set of all vectors the transformation can possibly output, over every input.

It has a name you already know. The output of M times any input is always a linear combination of the columns of M (that was the whole content of the matrix-vector product). So the set of all outputs is the span of the columns, the column space, the same span idea from the spans lesson, now applied to the columns of a matrix.

For an invertible 2x2 matrix, the two columns point in different directions, so their span is all of 2D: the transformation can reach every point in the plane. For a collapsed matrix, the columns are linearly dependent and lie on one line, so the column space is just that line: those are the only outputs the transformation can produce. The dimension of the column space is called the rank. Full 2D output is rank 2; a collapse to a line is rank 1; everything crushed to the origin is rank 0. Rank answers “how many dimensions of output does this transformation actually use?”

Null space: everything crushed to zero

The second idea is the null space: the set of all input vectors that the transformation sends to the zero vector. These are the inputs that get annihilated, the vectors that M maps to zero.

For an invertible transformation, the only vector that lands on the origin is the origin itself. Nothing else gets crushed, because nothing gets crushed at all. The null space is just the origin, the single point.

For a collapsed transformation, an entire line of inputs gets flattened onto the origin, and that line is the null space. It tells you precisely which direction the transformation destroyed: every vector along the null-space line was squashed to nothing, which is exactly why the transformation cannot be undone. The null space is the lost information, named.

A collapsed transformation has two lines that name what it does. The null space is the line of inputs that get crushed to the origin; the column space is the line that all outputs fall on. One says "what disappears"; the other says "what survives".

Conservation: rank plus nullity

These two dimensions are not independent. For a 2x2 matrix, they always add up to the input dimension, 2:

rank + nullity = input dimension

(The dimension of the null space is the nullity.) If the transformation uses both output dimensions (rank 2), nothing is crushed (nullity 0). If it collapses output to a line (rank 1), exactly one dimension’s worth of input gets destroyed (nullity 1). The dimensions you keep plus the dimensions you lose always sum to the dimensions you started with. Nothing leaks; the accounting is exact.

The capability: three ways to spot invertibility

That gives you three equivalent tests for whether a matrix is invertible, and the point of this lesson is that they are the same question asked three ways:

Determinant. A nonzero determinant means invertible.
Column space / rank. Full rank (column space is the entire input dimension) means invertible.
Null space. Null space is only the origin (nothing but zero gets crushed) means invertible.

If any one holds, all three hold; if any one fails, all three fail. A collapse shows up as zero determinant, as deficient rank, and as a null space bigger than the origin, all at once, because they are three faces of the same fact.

Worked examples

An invertible matrix. Take M with first column 2, 1 and second column 1, 1. Its determinant is 2 times 1 minus 1 times 1, which is 1, nonzero, so it is invertible. The columns 2, 1 and 1, 1 point in different directions, so their span is all of 2D: column space is the full plane, rank 2. The only input it sends to the origin is the origin, so the null space is just the origin, nullity 0. Rank 2 plus nullity 0 is 2, and all three tests agree: this transformation can be undone.

A non-invertible matrix. Take M with first column 2, 1 and second column 4, 2 from last lesson. Its determinant is 2 times 2 minus 4 times 1, which is 0. The columns 2, 1 and 4, 2 are dependent (the second is 2 times the first), so the column space is just the line through 2, 1, rank 1. For the null space, solve for the vectors the transformation maps to zero: both rows give the x-coordinate plus twice the y-coordinate equals zero, so the x-coordinate is negative twice the y-coordinate, and the null space is the line through negative-2, 1, nullity 1. Rank 1 plus nullity 1 is 2. Every vector along the negative-2, 1 direction gets crushed to the origin, and that is the dimension this transformation threw away.

Solving a system. Use the invertible M with first column 2, 1 and second column 1, 1, and target the point 3, 2. Because M is invertible, there is exactly one input that lands on 3, 2. Solving the system “twice the x-coordinate plus the y-coordinate equals 3” and “the x-coordinate plus the y-coordinate equals 2” gives the x-coordinate equal to 1 and the y-coordinate equal to 1, and indeed M applied to the vector 1, 1 gives 3, 2. One target, one answer.

Now the collapsed M with first column 2, 4 and second column 1, 2. A target off the column-space line, like the point 3, 2, has no solution at all: the transformation can never output it, because every output lies on the line through 2, 1. A target on the line, like the point 4, 2, has infinitely many solutions: one particular input works (such as 2, 0), and adding any null-space vector to it works too, since the null-space part gets crushed to zero and does not change the output. No solution or infinitely many: that is the two-sided cost of a zero determinant.

Why this matters when you use AI

Rank is one of the most practically loaded ideas in machine learning, because it answers “how many dimensions is this matrix actually using?” A matrix can be large, a thousand rows by a thousand columns, and still have low rank, meaning it genuinely operates in far fewer dimensions than its size suggests. When that happens, the matrix is compressible: you can approximate it with much smaller pieces and lose almost nothing. A good deal of work on shrinking large models leans on the observation that their big weight matrices, or the updates to them, are effectively low rank.

The null space carries the mirror-image lesson. It is the set of input directions a transformation is blind to, the differences it cannot see because it maps them to the same place. A transformation with a nontrivial null space has thrown information away, which is sometimes a defect and sometimes a deliberate compression. Rank tells you what a transformation can express; the null space tells you what it can no longer tell apart.

Common pitfalls

Thinking the inverse is “one over the matrix.” The inverse M-inverse is not a reciprocal of numbers; it is the transformation that undoes M. The defining property is M-inverse times M equals I, the do-nothing transformation, not any kind of division.

Believing every system has a solution. When the matrix is not invertible, a system can have no solution (the target is unreachable) or infinitely many (the target is reachable, but the null space adds wiggle room). Exactly one solution is the special case, and it happens precisely when the matrix is invertible.

Confusing column space with null space. Column space lives in the output and is everything the transformation can produce. Null space lives in the input and is everything the transformation destroys. They are different spaces answering different questions; do not let the similar names blur them.

Treating rank as the size of the matrix. Rank is the number of output dimensions actually used, not the row or column count. A big matrix can have small rank. That gap between size and rank is exactly what makes some matrices compressible.

What you should remember

The inverse M-inverse undoes M, defined by M-inverse times M equals I. It exists exactly when the determinant of M is nonzero, because a collapse (zero determinant) loses information that no undo transformation could recover. When it exists, the system M times the unknown equals the target has the unique solution: the unknown equals M-inverse times the target.
Column space is everything the transformation can output (the span of its columns); rank is its dimension. Null space is everything the transformation crushes to zero. An invertible matrix has full rank and a null space of just the origin.
Three tests, one question. Nonzero determinant, full rank, and trivial null space are the same statement about invertibility, and rank plus nullity equals the input dimension always holds: the dimensions kept plus the dimensions lost equal the dimensions you started with.

To undo a transformation is to know that nothing was lost, and the determinant, the rank, and the null space are three ways of checking the same thing. When something was lost, the null space names the direction it went. The next lesson breaks the assumption that input and output even have the same number of dimensions, with matrices that are not square.