Practice: What deep learning adds
Self-check
Section titled “Self-check”Seven short questions. Try to answer each in your head (or on paper) before opening the collapsible. Active retrieval is where the learning sticks; rereading is comfortable but does much less.
1. In one sentence, what does the word “deep” actually add to “neural network”?
Show answer
It means many layers, stacked one after another, plus the training tricks that make many layers actually trainable. Deep learning is the same neural-network engine from the previous track, just made deep.
2. The core ideas behind neural networks existed in the 1980s. Why did deep learning only take off around 2012, not decades earlier?
Show answer
Three things had to arrive together, and none alone was enough: better algorithms (training tricks like ReLU and dropout that kept deep networks trainable), much more data (large labeled datasets), and much more compute (GPUs that train networks in parallel). The idea waited for the world to catch up.
3. What three ingredients combined at the 2012 AlexNet/ImageNet moment?
Show answer
Depth (better algorithms making many layers work), data (ImageNet’s roughly 1.2 million labeled images), and compute (GPUs). “Depth, data, and compute” is the trio, and almost everything since has been that same trio scaled up.
4. Why can a network with no hidden layer not separate the XOR (checkerboard) pattern, and what fixes it?
Show answer
With no hidden layer the network can only draw a single straight line, and no straight line separates a checkerboard layout (one misfit always lands on the wrong side). Adding even one hidden layer lets the network bend and combine lines into a shape that separates the corners. That jump from impossible to easy is depth earning its keep.
5. In one sentence, why does depth help?
Show answer
Each layer transforms the data into a new form, and stacking layers composes those transformations, so a deep network builds complex patterns out of many simple steps. Shallow networks see simple patterns; deep networks build complicated ones from simple parts.
6. Name the four problem shapes this track will tour, and the one-line idea behind each.
Show answer
Sequences (carry information across steps), images (look at local patches of pixels), generation (learn to produce new examples, not just classify), and decisions (act to earn reward, reinforcement learning). The same engine underlies all four; only the arrangement changes.
7. Fill in the blank. “Deep learning is powerful ______-matching, not ______.”
Show answer
Pattern-matching, not understanding (and not AGI, and not “a brain in a computer”). It is genuinely powerful and genuinely bounded at the same time; both halves are true.
Try it yourself: make depth matter with your own eyes
Section titled “Try it yourself: make depth matter with your own eyes”This is the XOR demonstration from the lesson, turned into something you can run. About 10 minutes, no setup, no account.
Side effects: none. TensorFlow Playground runs entirely in your browser. No API calls, no costs.
Setup: open TensorFlow Playground in a new tab. On the left, under “DATA,” pick the XOR dataset (the one with two diagonal clusters of each color, a checkerboard). Leave the inputs as the two raw features X1 and X2.
Step 1: no hidden layer. Remove all hidden layers (use the minus button until there are zero). Press play and let it train. Watch the “Test loss” and the background classification.
What you’ll see
It cannot do it. With no hidden layer the network draws a single straight dividing line, and the loss stays high because no straight line separates a checkerboard. The background never matches the data. This is the XOR wall from the lesson, live.
Step 2: add one hidden layer. Add a hidden layer with about four neurons. Press play again.
What you’ll see
Now it works. The loss drops and the background bends into a shape that cleanly separates the diagonal clusters. One hidden layer was the difference between impossible and easy. You just watched depth earn its keep on the exact pattern from the lesson.
Step 3: play. Try more layers and more neurons, and try the harder “spiral” dataset. Notice that harder patterns need more depth and width to separate. (Do not worry about the learning-rate and activation controls yet; just feel how capacity relates to the complexity of the pattern.)
Flashcards
Section titled “Flashcards”Eleven cards. Click any card to reveal the answer. Use the Print flashcards button to lay out the full set as one card per page, ready to print or save as a PDF for offline review.
Q. What is deep learning, in one sentence?
Neural networks with many layers, plus the training tricks that make many layers trainable. The same engine from the previous track, made deep.
Q. Why did deep learning take off around 2012 and not in the 1980s?
Three things finally arrived together: better algorithms (ReLU, dropout), much more data, and much more compute (GPUs). The ideas were old; the conditions were new.
Q. What is the depth-data-compute trio?
The three ingredients whose combination unlocked modern deep learning at the 2012 AlexNet/ImageNet moment. Scaling this trio up is most of what “progress” has meant since.
Q. What was the significance of AlexNet in 2012?
A deep network that won the ImageNet image-recognition contest by a wide margin, marking the start of the modern era. About 60 million parameters across 8 layers, enormous for its time.
Q. Why can't a no-hidden-layer network learn XOR?
It can only draw a single straight line, and no straight line separates a checkerboard pattern. One hidden layer lets it bend and combine lines to separate the classes.
Q. Why does depth help?
Each layer composes a new transformation on the previous one, so a deep network builds complex patterns out of many simple steps.
Q. What are the four problem shapes this track tours?
Sequences (carry info across steps), images (local patches), generation (produce new examples), and decisions (act for reward). One engine, four arrangements.
Q. Is deep learning artificial general intelligence?
No. It is a powerful framework for learning patterns from examples, not a general mind and not “a brain in a computer” despite the borrowed word “neuron.”
Q. Is more depth always better?
No. Depth helps up to a point and brings its own costs (harder to train, more data needed). It fits some problems; it is not a dial you crank to infinity.
Q. What four questions can you ask of any deep-learning system?
What shape of problem does it solve, how is the engine wired for it, what was it trained on, and where will it break?
Q. What is the one-sentence takeaway of this lesson?
Deep learning is the neural network you already understand, made deep, and finally given the data and compute to show what depth was always capable of.