Practice: Eigenvectors and eigenvalues

Self-check

Six short questions. Answer each one in your head (or on paper) before opening the collapsible. Trying to retrieve the answer is where the learning sticks; rereading feels productive but does much less.

1. What is an eigenvector, and what is an eigenvalue?

Show answer

An eigenvector is a nonzero vector that the transformation only scales, without knocking it off its line: M · v = λ · v. The eigenvalue λ is the factor it gets scaled by. For an eigenvector, the whole matrix acts like a single number.

2. How do you find a matrix’s eigenvalues?

Show answer

Solve the characteristic equation det(M - λI) = 0. Subtract λ down the diagonal of M, take the determinant, set it to zero, and solve for λ.

3. Once you have an eigenvalue λ, how do you find its eigenvectors?

Show answer

Find the null space of (M - λI): the nonzero vectors that (M - λI) sends to zero. Those are the eigenvectors for that λ. (They form a line, or more, through the origin.)

4. Why does det(M - λI) = 0 give the eigenvalues?

Show answer

M · v = λ · v rearranges to (M - λI) · v = 0. A nonzero v is crushed to zero only when (M - λI) collapses space, which happens exactly when its determinant is zero (the inverses-lesson condition). So the eigenvalues are the λ that make the determinant vanish.

5. What does the eigenvector basis give you, and how?

Show answer

A diagonal matrix. With the eigenvectors as the columns of P, the change-of-basis sandwich D = P^-1 · M · P is diagonal with the eigenvalues on the diagonal. In the eigenvector basis the transformation is pure scaling along the axes, the simplest it can look.

6. Does every 2x2 matrix have two real eigenvector directions?

Show answer

No. A pure rotation has no real eigenvectors (every vector rotates off its line). A shear has only one. A full set of independent eigenvectors is common but not guaranteed; the characteristic equation can have no real roots or a repeated root.

Try it yourself, part 1: find eigenvalues and eigenvectors

For each matrix, solve det(M - λI) = 0 for the eigenvalues, find an eigenvector for each, and verify with M · v = λ · v. About 9 minutes, pen and paper.

a) [[4, 1], [0, 3]]
b) [[0, 2], [2, 0]]

Check your work

a) M - λI = [[4-λ, 1], [0, 3-λ]], det = (4-λ)(3-λ) = 0, so λ = 4 and λ = 3. For λ = 4: M - 4I = [[0, 1], [0, -1]], both rows give y = 0, eigenvector [1, 0] (check: M · [1, 0] = [4, 0] = 4·[1, 0]). For λ = 3: M - 3I = [[1, 1], [0, 0]], top row gives x + y = 0, eigenvector [1, -1] (check: M · [1, -1] = [3, -3] = 3·[1, -1]).
b) M - λI = [[-λ, 2], [2, -λ]], det = λ² - 4 = 0, so λ = 2 and λ = -2. For λ = 2: M - 2I = [[-2, 2], [2, -2]], gives y = x, eigenvector [1, 1] (check: M · [1, 1] = [2, 2] = 2·[1, 1]). For λ = -2: M + 2I = [[2, 2], [2, 2]], gives y = -x, eigenvector [1, -1] (check: M · [1, -1] = [-2, 2] = -2·[1, -1]). The negative eigenvalue flips the vector while scaling it.

Try it yourself, part 2: spot by eye, then diagonalize

About 7 minutes.

Step 1. Read the eigenvectors and eigenvalues of [[5, 0], [0, 2]] by eye, without solving anything. Why is it immediate?

Step 2. Diagonalize M = [[4, 1], [0, 3]] from part 1a. Build P from its eigenvectors and compute D = P^-1 · M · P. (Hint: P = [[1, 1], [0, -1]] is its own inverse, det(P) = -1.)

Check your work

Step 1. It is diagonal, so it stretches each axis independently: i-hat = [1, 0] is an eigenvector with eigenvalue 5, and j-hat = [0, 1] is an eigenvector with eigenvalue 2. For a diagonal matrix the standard basis is already the eigenvector basis, and the eigenvalues sit on the diagonal.

Step 2. Columns of P are the eigenvectors [1, 0] and [1, -1], and P^-1 = [[1, 1], [0, -1]] = P. First M · P: M · [1, 0] = [4, 0], M · [1, -1] = [3, -3], so M · P = [[4, 3], [0, -3]]. Then P^-1 · (M · P): P^-1 · [4, 0] = [4, 0] and P^-1 · [3, -3] = [3 - 3, 0 + 3] = [0, 3]. So D = [[4, 0], [0, 3]], diagonal with the eigenvalues 4 and 3 on it. The off-diagonal 1 in the original M was just an artifact of the standard basis.

Flashcards

Ten cards. Click any card to reveal the answer. Use the Print flashcards button to lay out the full set as one card per page, ready to print or save as a PDF for offline review.

Q. What is an eigenvector and an eigenvalue?

An eigenvector is a nonzero vector the transformation only scales, not rotates off its line: M · v = λ · v. The eigenvalue λ is the scaling factor. For an eigenvector, the whole matrix acts like a single number.

Q. How do you find a matrix's eigenvalues?

Solve the characteristic equation det(M - λI) = 0: subtract λ down the diagonal, take the determinant, set it to zero, and solve for λ.

Q. How do you find the eigenvectors for a given eigenvalue?

Find the null space of (M - λI): the nonzero vectors it sends to zero. Those are the eigenvectors for that λ, forming a line (or more) through the origin.

Q. Why does det(M - λI) = 0 give the eigenvalues?

M·v = λ·v becomes (M - λI)·v = 0. A nonzero v maps to zero only when (M - λI) collapses space, i.e. its determinant is zero. So the eigenvalues are the λ that make the determinant vanish.

Q. What does the eigenvector basis give you?

A diagonal matrix. With eigenvectors as the columns of P, the sandwich D = P^-1 · M · P is diagonal with the eigenvalues on it. In that basis the transformation is pure scaling along the axes, the simplest it can look.

Q. Does every 2x2 matrix have two real eigenvector directions?

No. A pure rotation has none (everything rotates off its line); a shear has only one. A full set of independent eigenvectors is common but not guaranteed.

Q. What does a zero eigenvalue mean?

That the transformation crushes that eigenvector’s direction to the origin: the collapse from the determinant and inverses lessons. A zero eigenvalue means the matrix is not invertible.

Q. Are eigenvectors single arrows or whole lines?

Whole lines through the origin. If v is an eigenvector, so is any nonzero scalar multiple of it (scaling does not change its line). People usually name one representative vector on the line.

Q. What is the difference between an eigenvalue and an eigenvector?

The eigenvalue is the number (the scaling factor λ); the eigenvector is the direction that gets scaled (v). In M · v = λ · v, v is the vector and λ is the number.

Q. How do eigenvalues explain exploding and vanishing gradients?

A signal passing repeatedly through a weight matrix is scaled by its eigenvalues each pass. Eigenvalues with magnitude above 1 make it blow up over many steps; below 1 make it decay to nothing. Training stability depends on keeping those magnitudes near 1.