Summary: Where deep learning breaks

Every lesson in this track so far has shown what deep learning can do. This one is the honest counterweight: where it breaks, and why. The lesson names four real limitations, data hunger, brittleness, the slant it inherits from its data, and the lack of guarantees, and connects each one back to the same mechanism the whole track has built: learning patterns from examples. These are not bugs someone forgot to fix; they are consequences of the approach. The point is not to dampen the wonder but to let you use these tools with clear eyes.

Core ideas

• It is hungry for data. Learning patterns purely from examples is the only thing it does, so it typically needs far more labeled data than seems reasonable. A child learns a giraffe from one picture book; a network may need thousands of images, and with too few it memorizes rather than generalizes. A poor fit where data is scarce or expensive.
• It is brittle at the edges. It matches patterns, it does not understand, so it can fail confidently on inputs unlike its training. The sharpest demonstration is the adversarial example: a tiny, human-invisible bit of noise flips a confident “panda” into a confident “gibbon.” The same fragility hits out-of-distribution inputs, and the model gives an answer rather than signaling doubt.
• It inherits the slant of its data. A model learns the imbalances in its training data along with everything else, so it is simply more accurate on what it saw more of. A skewed sample produces uneven performance, strong on common cases and weak on rare ones. This is a technical property of learning from examples, not intent, and more data helps only if it is more representative.
• It cannot guarantee or fully explain itself. Behavior lives in millions of opaque weights no person chose or can easily read. So there are no guarantees (it cannot certify an output is correct, and its confidence is unreliable) and there is opacity (it is hard to say why it decided what it did). For generative models these combine into hallucination: fluent, confident, fabricated output, because the model produces what looks like its training data, not output checked against truth.

What changes for you

These four limits are exactly the failures you meet using AI tools. The assistant that invents a citation is hallucinating. The classifier weirdly confident about a strange photo is failing out of distribution. The tool that works for common requests and stumbles on unusual ones is showing its data hunger and its data’s slant. Knowing the limits turns baffling glitches into expected behavior you can plan around: be skeptical of confident output, verify anything stated as fact, and never read confidence as a guarantee. A fair caveat: researchers work hard on all four and the situation improves, but these are deep properties of learning from data, so treat any claim that one is “solved” with the same skepticism. Next, the final lesson pulls the whole track together into one map of the field and points you toward where to go from here.