Summary: What deep learning adds

Deep learning is the neural network from the previous track, made deep: many layers stacked together, plus the training tricks that make many layers work. The puzzle this opening lesson answers is why an idea studied since the 1980s only took over technology around 2012. The short answer is that three things finally arrived together (depth, data, and compute), and almost everything since has been that trio scaled up. This summary is the scan-it-in-five-minutes version; the lesson builds the intuition and pays off the “why now” story.

Core ideas

• “Deep” means many layers. Deep learning is not a different idea from neural networks; it is neural networks with many layers, plus the tricks that make depth trainable. You already know the engine (neurons, weights, gradient descent, backpropagation); this track tours the vehicles built around it.
• The ideas are old; the conditions are new. Neural networks underperformed for roughly thirty years (the “AI winters”) because deep networks were hard to train: the training signal faded through many layers, datasets were too small, and compute was too slow. They worked on toy problems and lost to simpler methods on real ones.
• 2012 was the unlock, and it took three things at once. AlexNet won the ImageNet contest by a wide margin because better algorithms (ReLU, dropout), more data (about 1.2 million labeled images), and more compute (GPUs) arrived together. No single one would have done it. AlexNet had about 60 million parameters across 8 layers; today’s largest models reach hundreds of billions. Same idea, scaled.
• Depth lets a network build complex patterns from simple ones. The XOR example is the proof: a network with no hidden layer can only draw a single straight line and cannot separate a checkerboard, while one hidden layer can. Each layer composes a transformation on the last, so depth turns simple patterns into intricate ones.
• One engine, four problem shapes. The rest of the track surveys how the same engine is wired differently to fit different data: sequences (carry information across steps), images (local patches of pixels), generation (produce new examples), and decisions (reinforcement learning). Only the arrangement changes.
• Powerful and bounded, both at once. Deep learning excels at perception, generation, and large-scale pattern-finding, and it is genuinely limited: hungry for data, brittle on inputs unlike its training, and unable to guarantee when it is wrong. It is pattern-matching at scale, not understanding. The limits get their own lesson in Phase 3.

What changes for you

Before this lesson, the recent explosion of AI can look like a sudden leap in machine cleverness. After it, you can see it for what it is: an old idea finally given enough depth, data, and compute to show its range. That reframing lets you read AI news with sharper questions, what was this trained on, how big is it, where would it break, and stop being surprised by the field’s split personality (dazzling on perception and generation, clumsy on anything needing real reasoning or guarantees). The next lesson begins the tour with the first problem shape, sequences: why a plain feedforward network struggles with ordered data, and how giving a network memory fixes it.

Deep learning is not a new kind of thinking. It is the neural network you already understand, made deep, and finally given the data and the compute to show what depth was always capable of.