Dot products and projection
What you’ll learn
Section titled “What you’ll learn”This is lesson 9 of Track 4 (Visual Math: Linear Algebra), and it closes Phase 2 by cashing a promise the very first lesson made: that AI compares vectors using the dot product. By the end you will be able to compute a dot product two ways, by multiplying matching components and adding, and by multiplying lengths and the cosine of the angle, and explain why these two unrelated-looking formulas always agree. You will read the sign as a measure of agreement (same direction, perpendicular, or opposing), see dotting with a unit vector as a projection, and meet the duality idea that ties it all together: a vector is secretly a transformation that turns other vectors into numbers. The source curriculum is 3Blue1Brown’s Essence of Linear Algebra by Grant Sanderson, freely available at 3blue1brown.com.
Where this fits
Section titled “Where this fits”This is lesson 9 of 15, the final lesson of Phase 2 (geometry of operations). It is the direct payoff of Nonsquare matrices: the 1-row matrix from that lesson (a transformation from many dimensions down to a single number) turns out to be a vector you dot against. It also closes a loop opened in lesson 1, which named the dot product as the operation behind attention without yet defining it. The next phase (advanced perspectives) opens with the cross product, an operation that takes two vectors and returns a third.
Before you start
Section titled “Before you start”Prerequisites: Nonsquare matrices (the 1-row matrix as a transformation to a single number), and comfort with vector length and the basic trigonometry of cosine. The duality argument leans directly on the previous lesson, so it helps to have that fresh. The practice is pen and paper; a calculator is handy for one cosine but not required.
About the math
Section titled “About the math”There are two formulas and one small piece of trigonometry (the cosine of an angle). The practice has you compute dot products by the algebraic formula, read off the sign, and use the geometric formula once to recover a cosine (which is exactly cosine similarity). The arithmetic is light; the depth is conceptual, in seeing why the coordinate formula and the angle formula are the same number.
By the end, you’ll be able to
Section titled “By the end, you’ll be able to”- Compute a dot product with the algebraic formula (multiply matching components and add)
- Interpret the geometric formula |v||w|cos(theta) and read the sign as same-direction, perpendicular, or opposing
- Explain dotting with a unit vector as the signed length of a projection
- Explain via duality why the two formulas agree (a vector is a 1-row matrix lying on its side)
- Connect the dot product to attention scores, cosine similarity, and a single neuron
Time and difficulty
Section titled “Time and difficulty”- Read time: about 11 minutes
- Practice time: about 15 minutes (a compute-and-read-the-sign exercise, a both-formulas cosine-similarity drill, and flashcards)
- Difficulty: standard (a Phase 2 lesson; light arithmetic, with the depth in the duality argument)