Cheatsheet: Dot products and projection

Two formulas, same number

Formula	Expression
Algebraic	v · w = v1·w1 + v2·w2 + ... + vn·wn (multiply matching components, sum)
Geometric	`v · w =

Output is a single number, not a vector. The two formulas always agree.

The sign tells the story

Sign	Angle	Meaning
Positive	less than 90 deg	Vectors broadly point the same way
Zero	exactly 90 deg	Vectors are perpendicular
Negative	more than 90 deg	Vectors broadly oppose

Projection

For a unit vector u-hat (length 1):

v · u-hat = |v| · cos(θ) = signed length of v's projection onto the u-hat line

Drop a perpendicular from the tip of v onto u-hat’s line; the origin-to-foot distance (with sign) is the dot product. “How far does v reach in this direction?”

Duality (why the formulas agree)

A 1-row matrix [a b] applied to [x, y] gives a·x + b·y, which is exactly [a, b] · [x, y]. So dotting with a vector = applying the 1-row matrix that is the vector lying on its side. A vector and a “vector-to-number” transformation are the same object.

Worked examples

Dot product	Algebraic	Geometric
[3,4] · [1,0]	3 + 0 = 3	projection onto x-axis = 3
[1,1] · [1,-1]	1 - 1 = 0	perpendicular (90 deg)
[3,4] · [1,0]	3	5·1·cos θ = 3 so cos θ = 3/5
[1,0] · [-1,1]	-1 + 0 = -1	135 deg, 1·√2·cos135 ≈ -1

Commutative: v · w = w · v (same sum; same angle).

Why it matters for AI

• Attention: relevance of one token to another is query · key. Bigger dot product = more attention.
• Cosine similarity: dot of two unit vectors = cos(θ), the standard “how similar are these embeddings” measure (search, clustering, retrieval).
• A neuron: computes weight · input, then a nonlinearity. “How much does the input point along my direction?”

Pitfalls to dodge

• Expecting a vector back. Output is one number. (A vector-out operation is the cross product, coming up.)
• Ignoring the sign. Negative is information: broadly opposite directions.
• Reading it as distance. It measures shared direction scaled by lengths, not how far apart.
• Projection without a unit vector. v · u-hat is the projection length only when |u-hat| = 1.

The one-line version

A dot product is one number answering “how much do these two vectors point the same way?”, computable from coordinates or from an angle, because a vector is a transformation in disguise.