Practice: Neurons as numbers, layers as structure
Self-check
Section titled “Self-check”Six short questions. Answer each one in your head (or on paper) before opening the collapsible. Trying to retrieve the answer is where the learning sticks; rereading feels productive but does much less.
1. In a neural network, what is a neuron, and what is the name of the number it holds?
Show answer
A neuron is just a container that holds a single number between 0 and 1. That is the whole definition: not a cell, not a switch, not a little brain. The number it holds is called its activation. Near 1, the neuron is “lit up”; near 0, it is quiet; in between is partly lit.
2. How many neurons are in the input layer for a 28 by 28 image, and what does each one hold?
Show answer
784 neurons, one per pixel (28 times 28 = 784). Each neuron’s activation is set to the brightness of its pixel: 0 for fully black, 1 for fully white, and a value in between for gray. The whole image enters the network as 784 numbers in 784 boxes.
3. The output layer has ten neurons. How do you read the network’s answer off them?
Show answer
Each output neuron holds an activation you read as a confidence score for one digit (0 through 9). The network’s guess is simply the digit whose neuron has the highest activation. A clear winner means confidence; two near-equal top values mean the network is hesitating between two digits.
4. What does “hidden layer” actually mean, and what do hidden layers do?
Show answer
“Hidden” just means “not the input layer and not the output layer.” The hidden layers sit in between and do the work of turning raw pixel brightness into a digit guess. In the example network there are two hidden layers of 16 neurons each, but those exact numbers are a design choice, not a rule.
5. Why is this kind of network called “feedforward”?
Show answer
Because the numbers flow in a single direction: input layer, into the first hidden layer, into the second, out through the output layer. Forward, always forward, with no loops and no later layer feeding back into an earlier one. Each layer takes the previous layer’s activations and produces the next layer’s.
6. Someone says: “The first hidden layer definitely detects edges and the second definitely detects loops.” What is the honest correction?
Show answer
That is the hope for what hidden layers might do, and a useful first picture, but it is not a proven fact. A real trained network does not reliably organize itself that neatly; the patterns it learns tend to be messier and less human-readable. Hold the edges-to-loops story as motivation, not as a description of what is provably happening inside.
Try it yourself, part 1: load a pixel, read a guess
Section titled “Try it yourself, part 1: load a pixel, read a guess”Pen and paper, about 8 minutes. This is the lesson’s two core moves, done with your own hands: putting a pixel into the input layer, and reading an answer off the output layer.
Setup. A different, smaller image this time: 16 pixels wide by 16 pixels tall. Neurons are numbered row by row, the same way as in the lesson, so the neuron number for a pixel is row times width + column.
Step 1. A pixel sits at row 5, column 9, with brightness 0.3. Two questions: how many neurons does this 16 by 16 input layer have in total, and which neuron number does this pixel land in, holding what activation?
Step 2. The network finishes and its ten output neurons (digits 0 to 9) hold these activations:
0:0.01 1:0.02 2:0.03 3:0.04 4:0.025:0.01 6:0.03 7:0.81 8:0.02 9:0.01What is the network’s guess, and is it confident or hesitating?
Step 3. A second image gives this output instead:
0:0.02 1:0.46 2:0.03 3:0.01 4:0.025:0.01 6:0.01 7:0.44 8:0.00 9:0.00What is the guess now, and what is different about how sure the network is?
Show answer
Step 1. A 16 by 16 input layer has 16 times 16 = 256 neurons. The pixel at row 5, column 9 lands in neuron number 5 times 16 + 9 = 89, and that neuron holds the activation 0.3 (its pixel’s brightness).
Step 2. Scan for the tallest. Digit 7 holds 0.81, far above every other neuron, so the guess is 7, and the high value means the network is confident.
Step 3. The tallest is digit 1 at 0.46, so the guess is 1, but only barely: digit 7 is right behind it at 0.44. The network is hesitating between 1 and 7, which makes sense, since a 1 and a 7 can look very alike. Same reading rule (tallest wins), but a near-tie tells you the answer is shaky, not certain.
Try it yourself, part 2: count the neurons
Section titled “Try it yourself, part 2: count the neurons”About 3 minutes, arithmetic only. You design a small digit recognizer with this structure:
- Input layer: one neuron per pixel of a 16 by 16 image
- Hidden layer 1: 20 neurons
- Hidden layer 2: 20 neurons
- Output layer: one neuron per digit
How many neurons does the whole network have?
Show answer
Add the layers: input is 16 times 16 = 256, then 20, then 20, then 10 for the output (one per digit 0 to 9). Total = 256 + 20 + 20 + 10 = 306 neurons. Notice the input layer dominates the count, exactly as it does in the lesson’s 826-neuron example, because there is one input neuron for every single pixel.
Flashcards
Section titled “Flashcards”Nine cards. Click any card to reveal the answer. Use the Print flashcards button to lay out the full set as one card per page, ready to print or save as a PDF for offline review.
Q. What is a neuron in a neural network?
A container that holds a single number between 0 and 1. Not a cell, not a switch, not a little brain. Just a box with one number in it.
Q. What is a neuron's activation?
The number a neuron holds, from 0 to 1. Near 1 the neuron is “lit up” or “firing”; near 0 it is quiet; in between is partly lit.
Q. What is the input layer, and how big is it for a 28 by 28 image?
The layer where the image comes in, with one neuron per pixel. A 28 by 28 image gives 784 input neurons, each holding its pixel’s brightness (0 = black, 1 = white).
Q. What is the output layer, and how do you read the answer?
One neuron per possible answer (10 neurons for digits 0 to 9). Each holds a confidence score; the digit whose neuron has the highest activation is the network’s guess.
Q. What is a hidden layer?
Any layer that is not the input or the output. Hidden layers sit in between and do the work of turning raw pixel brightness into a digit guess.
Q. What does 'feedforward' mean?
The numbers flow in one direction only, from input through the hidden layers to the output. No loops, no later layer feeding back into an earlier one.
Q. How many neurons are in the example network, and what is its shape?
784 (input) + 16 + 16 + 10 (output) = 826 neurons. The two hidden layers of 16 are a clean teaching choice, not a rule; real networks vary widely.
Q. Where does a pixel at row r, column c land in the input layer?
At neuron number r times width + c, numbering row by row. For a 28-wide image, row 10 column 14 is neuron 10 times 28 + 14 = 294, holding that pixel’s brightness.
Q. Do hidden layers really detect edges then loops?
That is the hope and a useful first picture, but not a proven fact. A trained network’s real patterns are messier and less human-readable. Hold the edges-to-loops story as motivation, not description.