Cheatsheet: Where deep learning breaks
The one idea that matters
Section titled “The one idea that matters”Every prior lesson: "here's what deep learning can do."This one: "here's what it can't, and why."Each limitation falls out of HOW these systems work, not from a missing patch.The four limitations
Section titled “The four limitations”| Limitation | What it is | Why (connects to) | Why it matters |
|---|---|---|---|
| Data hunger | needs lots of labeled data | learning purely from examples (L1’s data pillar) | poor fit when data is scarce; small data → memorizing |
| Brittleness | confident failure on inputs unlike training | pattern-matching, not understanding | adversarial + out-of-distribution inputs break it |
| Data-slant (bias) | mirrors imbalances in its data | learns whatever the data contains | worse on under-represented cases |
| No guarantees / opacity | can’t certify correctness; hard to explain | knowledge is in opaque weights (T11) | confidence ≠ correctness; “black box” |
Brittleness, sharp version
Section titled “Brittleness, sharp version”Adversarial example: a panda image, correctly classified. Add a tiny, human-invisible noise → the network says “gibbon,” confidently. It keyed on fragile statistics, not the concept. Same fragility on out-of-distribution inputs (unusual angles, unseen topics): it answers anyway, often confidently.
Bias = a mirror, technically
Section titled “Bias = a mirror, technically”A model is more accurate on what it saw more of. Over-represent some cases and under-represent others → uneven performance. This is a statistical property of learning from data, not intent. More data fixes it only if the data is more representative.
Hallucination (generative models)
Section titled “Hallucination (generative models)”A text generator can produce fluent, confident, fabricated output (a citation to a paper that does not exist), because it produces what looks like its training data, not what is checked against truth. Convincing ≠ correct.
Why it matters when you use AI
Section titled “Why it matters when you use AI”- Invented citation = hallucination.
- Weird confidence on a strange photo = out-of-distribution failure.
- Smooth on common requests, stumbles on rare ones = data hunger + slant.
- Rules of thumb: be skeptical of confident output (especially on novel/rare inputs); verify anything stated as fact; never read confidence as a guarantee.
Honest caveat
Section titled “Honest caveat”Researchers work on all four and things improve, but they are deep properties of learning from data, not quick bugs. Treat “this is solved” claims with the same skepticism.
Pitfalls to dodge
Section titled “Pitfalls to dodge”- “These are temporary bugs.” No. Consequences of the approach; improved by research, not patched away.
- “Trust the model’s confidence.” No. Confidently wrong is a thing (panda→gibbon, invented citation).
- “More data fixes bias.” Only if it is more representative data.
- “A fluent answer is a true answer.” No. Generative models optimize for plausible, not verified.
The one-line version
Section titled “The one-line version”Everything in this track is real, powerful, and bounded; understanding deep learning means seeing why the same machinery produces both the wins and the failures.