Practice: Dot products and projection
Self-check
Section titled “Self-check”Six short questions. Answer each one in your head (or on paper) before opening the collapsible. Trying to retrieve the answer is where the learning sticks; rereading feels productive but does much less.
1. What are the two formulas for the dot product?
Show answer
Algebraic: multiply matching components and add them, v1·w1 + v2·w2 + ... + vn·wn. Geometric: |v| · |w| · cos(θ), the two lengths times the cosine of the angle between them. They always give the same single number.
2. What does the sign of a dot product tell you?
Show answer
How the two vectors are oriented relative to each other. Positive means they broadly point the same way (angle under 90 degrees); zero means they are perpendicular (exactly 90 degrees); negative means they broadly oppose (angle over 90 degrees).
3. What does dotting a vector with a unit vector compute?
Show answer
The signed length of the projection of v onto the line through the unit vector: v · u-hat = |v| · cos(θ). It answers “how far does v reach in the u-hat direction?” Positive if v leans the same way as u-hat, negative if opposite. (This only equals the projection length when the second vector has length 1.)
4. What is the duality insight that explains why the two formulas agree?
Show answer
A 1-row matrix [a b] applied to [x, y] gives a·x + b·y, which is exactly [a, b] · [x, y]. So dotting with a vector is the same as applying the 1-row matrix that is the vector lying on its side. A vector and a “vector-to-number” transformation are the same object; the algebraic formula is the computation, the geometric formula is what that transformation does to space (project and scale).
5. Is the dot product commutative?
Show answer
Yes: v · w = w · v. Algebraically the sum of matching products is the same either way; geometrically, both equal |v|·|w|·cos(θ), and the angle between two vectors does not care which one you name first.
6. Does the dot product return a vector or a number?
Show answer
A single number (a scalar), not a vector. An operation on two vectors that returns a third vector is the cross product, which comes up in the next lessons.
Try it yourself, part 1: compute and read the sign
Section titled “Try it yourself, part 1: compute and read the sign”For each pair, compute the dot product with the algebraic formula, then say what the sign means (same direction, perpendicular, or opposing). About 6 minutes, pen and paper.
- a)
[2, 3] · [4, 1] - b)
[2, 1] · [-1, 2] - c)
[3, 0] · [-2, 5] - d)
[1, 2, 2] · [2, 1, -2]
Check your work
- a)
(2)(4) + (3)(1) = 8 + 3 = 11. Positive: the vectors broadly point the same way (angle under 90 degrees). - b)
(2)(-1) + (1)(2) = -2 + 2 = 0. Zero: the vectors are perpendicular. - c)
(3)(-2) + (0)(5) = -6 + 0 = -6. Negative: the vectors broadly oppose (angle over 90 degrees). - d)
(1)(2) + (2)(1) + (2)(-2) = 2 + 2 - 4 = 0. Zero: perpendicular, and the formula works the same in 3D (just one more term).
Try it yourself, part 2: both formulas and cosine similarity
Section titled “Try it yourself, part 2: both formulas and cosine similarity”About 8 minutes, pen and paper.
Step 1. Compute [3, 4] · [4, 3] with the algebraic formula. Then, given that |[3, 4]| = 5 and |[4, 3]| = 5, use the geometric formula v · w = |v|·|w|·cos(θ) to find cos(θ).
Step 2. The vectors u = [0.6, 0.8] and w = [0.8, 0.6] are both unit vectors (check: 0.36 + 0.64 = 1). Compute u · w. Since both have length 1, what does that number equal directly, and how does it compare to Step 1?
Check your work
Step 1. [3, 4] · [4, 3] = (3)(4) + (4)(3) = 12 + 12 = 24. Geometric: 24 = 5 · 5 · cos(θ), so cos(θ) = 24/25 = 0.96. The two vectors are nearly aligned (about a 16-degree angle).
Step 2. u · w = (0.6)(0.8) + (0.8)(0.6) = 0.48 + 0.48 = 0.96. Because both u and w are unit vectors, the dot product equals cos(θ) directly. This is cosine similarity, and it matches Step 1 exactly, because u and w are just [3, 4] and [4, 3] scaled to length 1 ([3,4]/5 = [0.6, 0.8]). Dividing out the lengths leaves only the cosine.
Flashcards
Section titled “Flashcards”Ten cards. Click any card to reveal the answer. Use the Print flashcards button to lay out the full set as one card per page, ready to print or save as a PDF for offline review.
Q. What are the two formulas for the dot product?
Algebraic: v1·w1 + v2·w2 + ... + vn·wn (multiply matching components, sum). Geometric: |v|·|w|·cos(θ) (lengths times cosine of the angle). Both give the same single number.
Q. What does the sign of a dot product tell you?
The relative orientation: positive = broadly same direction (angle under 90 deg), zero = perpendicular (exactly 90 deg), negative = broadly opposite (angle over 90 deg). The sign is information, not an error.
Q. What does dotting a vector with a unit vector compute?
The signed length of the projection of v onto the unit vector’s line: v · u-hat = |v|·cos(θ). It answers “how far does v reach in that direction?”, positive or negative depending on which way v leans.
Q. What is the duality insight behind the dot product?
Dotting with a vector equals applying the 1-row matrix that is the vector lying on its side: [a b]·[x,y] = a·x + b·y = [a,b]·[x,y]. A vector and a vector-to-number transformation are the same object, which is why the two formulas agree.
Q. Is the dot product commutative?
Yes, v · w = w · v. The algebraic sum is the same either way, and geometrically both equal |v|·|w|·cos(θ), where the angle does not depend on which vector you name first.
Q. Does the dot product return a vector or a number?
A single number (a scalar). An operation that takes two vectors and returns a third vector is the cross product, not the dot product.
Q. What is cosine similarity?
The dot product of two unit vectors, which by the geometric formula equals cos(θ): the cosine of the angle between them. It is the standard measure of how similar two embeddings are (close angle = close meaning).
Q. How is the dot product used in attention?
An attention mechanism scores how relevant one token is to another by the dot product of a query vector with a key vector. A larger dot product means the vectors point more the same way, so the token gets more attention.
Q. What does a single neuron compute with the dot product?
The dot product of its weight vector with the input, followed by a nonlinearity. The weight vector is a direction the neuron cares about; the dot product measures how strongly the input points that way.
Q. Why is a zero dot product the test for perpendicular?
By the geometric formula v · w = |v|·|w|·cos(θ), and cos(90°) = 0. For nonzero vectors, the product is zero exactly when the cosine is zero, which is exactly when the angle is 90 degrees.