Lesson: Spans and basis

In the last lesson you ended up with two operations and nothing else: add two vectors, and scale a vector by a number. That felt almost too small to matter. This lesson is where it starts paying off.

Here is the question that runs the whole lesson. Pick a couple of vectors. Allow yourself to stretch them by any amount and add the results together, using only the two operations you already have. Which points in the plane can you reach? Not “which points are nearby,” but which points are reachable at all, given unlimited use of add and scale. The set of all those reachable points has a name, and that name, span, is one of the most useful words in linear algebra.

Start with one vector

Take a single nonzero vector, say the vector with components 2, 1. Scale it. Doubling gives 4, 2. Halving gives 1, 0.5. Multiplying by negative 3 gives negative 6, negative 3. Every scalar multiple lands somewhere, and if you collect all of them, every possible stretch, squish, and flip, they trace out a straight line through the origin.

Every scalar multiple of v lands somewhere on the same line through the origin. That line, all the scalar multiples taken together, is the span of v. The span of one nonzero vector is always a line.

That line is the span of a single vector. One vector, used with scaling alone, reaches a 1D line and nothing off it. (The one exception: the zero vector, the one whose components are both zero; scaling it always gives the zero vector again, so its span is just the single point at the origin.)

Add a second vector: the span

Now bring in a second vector and allow yourself to add. A linear combination of two vectors, the first and the second, is any expression of the form

a · v + b · w

where the two scalars are yours to choose freely. That is just the two operations from the last lesson used together: scale the first vector by the first scalar, scale the second vector by the second, add the results. The span of a set of vectors is the set of all linear combinations of them, every point you can reach by choosing every possible pair of scalars.

For two vectors in the plane, there are exactly three things the span can be.

The generic case: the whole plane. Pick two vectors that point in genuinely different directions, like the vectors 1, 0 and 0, 1, or almost any random pair. By dialing the two scalars up, down, and negative, you can reach every single point in 2D space. The span is the entire plane.

Two vectors pointing in different directions: by scaling each and adding, you can land on any point in the plane. The span is the whole 2D plane, shown tinted behind the arrows.

The lined-up case: just a line. If the two vectors happen to lie on the same line through the origin, meaning one is already a scalar multiple of the other, then adding scaled copies of them never gets you off that line. The second vector told you nothing new. The span collapses back to a single 1D line.

If w lies on the same line as v, the two vectors are collinear, and w adds no new direction. The span is still just one line; the second vector is redundant. This is the failure mode the basis idea is built to avoid.

The collapsed case: just the origin. If both vectors are the zero vector, every linear combination is still the origin. The span is one point.

The interesting boundary is between the first two cases. Two vectors fill the plane exactly when they point in different directions. The moment they line up, you lose a whole dimension of reach.

Basis vectors

When two vectors do span the full plane, they are doing something worth a name. Any pair of vectors whose span is the entire 2D space is called a basis for the plane.

The most familiar basis is the standard basis: i-hat, the vector 1, 0, pointing one unit right, and j-hat, the vector 0, 1, pointing one unit up. These are quietly what coordinates have meant all along. When you write the vector 3, 4, you are really saying “3 of i-hat plus 4 of j-hat,” which is the linear combination 3 times the vector 1, 0 plus 4 times the vector 0, 1. The coordinates are the amounts of each basis vector.

The two unit arrows along the axes, î and ĵ, are the standard basis. Any 2D vector is just so many î plus so many ĵ. The vector [3, 4] is three rightward steps and four upward steps; the brackets and the arrow are the same thing in two notations.

What makes a basis useful is that every vector in the space is a linear combination of the basis vectors in exactly one way. There is one and only one choice of the two scalars that produces any given target. That uniqueness is what lets coordinates be coordinates: no ambiguity about which numbers describe a point.

A combination, worked

Suppose you want to reach the point 5, 3. With the standard basis it is immediate, and the coordinates hand you the answer directly:

5 · [1, 0] + 3 · [0, 1] = [5, 3]   ->   a = 5,  b = 3

Now reach the same point with a different basis, the vectors 1, 1 and 1, negative-1. You need two scalars that combine them to land on the point 5, 3:

a · [1, 1] + b · [1, -1] = [5, 3]
component by component:  a + b = 5,   a - b = 3
solves to:  a = 4,  b = 1
check:  4 · [1, 1] + 1 · [1, -1] = [4, 4] + [1, -1] = [5, 3]

Same point, a different basis, different coordinates, and again exactly one answer.

Now contrast a dependent pair. Take the vectors 2, 1 and 4, 2. The second is just 2 times the first, so it sits on the same line. Try to reach the point 5, 3, which is off that line, and no choice of scalars works: every combination of the two lands back on the line through 2, 1. The span is a line, and the point 5, 3 is not on it. That is what dependence costs you, a whole dimension of reach.

Independence and redundancy

The lined-up case from earlier has a name too. A set of vectors is linearly dependent if at least one of them is already in the span of the others, meaning you could throw it away without shrinking the span. It is redundant. Two vectors on the same line are dependent: the second adds no reach the first did not already have.

A set is linearly independent when no vector is redundant, when every vector adds a genuinely new direction the others could not reach. Independence is exactly the property that separated the plane-filling pairs from the line-bound pairs above.

With that word in hand, the definition of a basis tightens into something clean: a basis of a space is a set of vectors that is linearly independent and spans the whole space. Independent means no redundancy. Spanning means no gaps. A basis is the smallest set of vectors that reaches everything, with nothing wasted.

Stepping up to 3D

The same story plays out one dimension higher, and it is worth seeing once so the pattern generalizes in your head.

One vector in 3D still spans a line. Two vectors that point in different directions span a flat plane through the origin, a 2D sheet floating inside 3D space. To fill all of 3D you need three vectors, and only if none of them lies in the plane spanned by the other two. If the third vector is already in that plane, it is redundant, the set is linearly dependent, and the span stays a flat plane instead of filling space.

Two vectors in 3D, if they point in different directions, sweep out a flat plane through the origin: every combination of v and w lands somewhere on that plane. Add a third vector that points off the plane, and combinations of all three reach every point in 3D space. The lift off the plane is what fills space.

The pattern is the one to carry forward: each genuinely independent vector you add lifts the span up by one dimension, until you run out of new directions.

Why this matters when you use AI

A model that represents words or images as vectors is working inside a space of some fixed dimension, often hundreds or thousands. That number, the dimension, is exactly the size of a basis for the space: how many independent directions it takes to describe any point. Every embedding the model stores is a linear combination of those basis directions, the same scale-each-and-add idea you just learned, with hundreds of terms instead of two.

Independence is not just bookkeeping here. When two directions in a representation are not independent, one of them is redundant, and the model is spending storage on a direction that adds no new reach. A good deal of practical work on making models smaller is, underneath, the search for a smaller basis that still spans almost everything the original did. Span tells you what a representation can express; independence tells you whether it is doing so without waste.

Common pitfalls

Thinking span means “the vectors themselves.” The span is not the two arrows you started with; it is the entire infinite set of points you can reach from them by adding and scaling. A pair of short vectors and a pair of long ones can have the exact same span.

Forgetting the origin is always in the span. Choosing both scalars equal to 0 gives the zero vector, so the origin is reachable from any set of vectors. Every span passes through the origin. This is why spans are lines and planes through the origin, never lines or planes off to the side.

Confusing dependent with useless. A linearly dependent vector is redundant for the purpose of span, but that does not make it meaningless. It just means it carries no direction the others lacked. Dependence is a statement about reach, not about value.

Assuming more vectors always means more span. Add a fourth vector to three that already span 3D, and the span does not grow, there is nowhere left to grow. Once you span the space, extra vectors are all dependent by definition.

What you should remember

The span of a set of vectors is everything you can reach from them with the two operations from the last lesson: scale each one, add the results. For two vectors in the plane it is either the whole plane, a single line, or just the origin.
A basis is a linearly independent set that spans the space: no redundancy, no gaps. The standard basis i-hat and j-hat is why the vector 3, 4 means “3 right and 4 up,” and every vector is a unique combination of basis vectors.
Independence is about new reach. A vector is redundant when it already sits in the span of the others. Each genuinely independent vector lifts the span by one dimension, and the dimension of a space is just the size of a basis for it.

Span is what you can build. A basis is the smallest kit that builds all of it. Dimension is the size of that kit. The next lesson turns these static pictures into motion, asking what happens when a single rule moves every vector in the space at once.