Practice: Teaching machines to imagine

Self-check

Six short questions. Try to answer each one in your head (or on paper) before opening the collapsible. Active retrieval is where the learning sticks; rereading feels productive but does much less.

1. In one sentence each, what does a discriminative model learn, and what does a generative model learn?

Show answer

A discriminative model learns the boundary between groups: given an input, produce a label. A generative model learns what the data itself looks like, well enough to produce brand-new examples of it.

2. You have a network that takes a chest X-ray and outputs “pneumonia” or “clear.” Which type is it, and how do you know?

Show answer

Discriminative. The giveaway is the job, not the subject matter: it takes an input and returns a judgment about it. It never needs to be able to draw an X-ray, only to tell one category from another.

3. An autoencoder is shaped like an hourglass. What is the narrow middle called, and what is the network forced to do because of it?

Show answer

The narrow middle is the latent code. Because it is narrow, the network is forced to pack the essence of the input into a short list of numbers and then unpack it again, keeping only what matters.

4. What single twist turns a plain autoencoder into a VAE, and why does that twist make generation possible?

Show answer

The VAE trains the latent space to be smooth and well-organized, with no holes, so every nearby point decodes into a plausible example. Once the space is organized that way, you can pick a brand-new point the network never saw, decode it, and get a new, plausible example. A plain autoencoder only rebuilds what it was given.

5. In a GAN, name the two networks, say what each is trying to do, and say what the generator learns from.

Show answer

The generator tries to produce fakes good enough to pass as real. The discriminator (a plain classifier) tries to tell the fakes from genuine examples. The generator never sees a real example directly; it learns entirely from whether it managed to fool the discriminator. Think counterfeiter versus detective, both improving round after round.

6. Why is “imagine” a slightly misleading word for what a generative model does?

Show answer

“Imagine” suggests understanding or conjuring from an inner idea. A generative model learned the statistical shape of its training data and samples from it. The results can be striking, but it is producing patterns like the ones it was trained on, not creating from comprehension.

Try it yourself: sort the systems, then explain a mechanism

No math here, just the two ideas put to work. About 15 minutes.

Side effects: none. This is a thinking-and-writing exercise. No tools, no API calls, no costs.

Part A: discriminative or generative?

For each system below, decide whether it is discriminative (judges an input) or generative (produces a new example), and write one phrase saying how you can tell.

A filter that flags incoming email as spam or not spam.
A system that writes a short product description from a few bullet points.
A model that listens to a voice clip and identifies which language is being spoken.
A tool that produces a new piece of music in the style of a given composer.
A network that tags incoming customer-support tickets by topic.
A system that turns a written description into a new image.

Show answer

Discriminative. Input in, label out (spam / not spam).
Generative. It produces a new example (text that did not exist before).
Discriminative. It returns a judgment about the input (which language).
Generative. It produces a new example (a new piece of music).
Discriminative. Input in, label out (which topic).
Generative. It produces a new example (an image).

The pattern: if the output is a verdict about the input, it is discriminative. If the output is a brand-new artifact, it is generative.

Part B: explain the GAN contest in three sentences.

Without looking back at the lesson, write a three-sentence explanation of how a GAN learns, as if to a curious friend. Use the counterfeiter-and-detective framing if it helps.

Show a model answer

A GAN trains two networks against each other: a generator that makes fakes and a discriminator that tries to catch them, like a counterfeiter and a detective. Every time the detective spots a fake, the counterfeiter learns and improves; every time a fake slips through, the detective sharpens up. The contest itself is the training, and as both escalate, the generator is driven to produce examples convincing enough to fool a detector that keeps getting better.

If your version captured “two networks competing,” “the generator learns only from the discriminator’s verdicts,” and “both improve through the contest,” you have the idea.

Flashcards

Ten cards. Click any card to reveal the answer. Use the Print flashcards button to lay out the full set as one card per page, ready to print or save as a PDF for offline review.

Q. What is the difference between a discriminative and a generative model, in one line each?

Discriminative: learns the boundary between groups (input → label). Generative: learns what the data itself looks like and produces new examples of it.

Q. What does a discriminative model need to know about a cat?

Only what tells a cat apart from the alternatives. It never needs to know what a cat fully looks like, just where the dividing line is.

Q. What does a generative model need to learn about a cat?

The shape of the whole category: the patterns, textures, and arrangements that make a cat a cat, well enough to produce a brand-new cat that was never in the training set.

Q. What shape is an autoencoder, and what does the shape force it to do?

An hourglass: input squeezed down through a narrow middle, then expanded back out. The narrow middle forces the network to pack the essence of the input into a short code and unpack it again.

Q. What is the latent code?

The short list of numbers in the narrow middle of an autoencoder: the compressed essence of an input.

Q. What twist makes a VAE able to generate, when a plain autoencoder cannot?

The VAE trains the latent space to be smooth and organized, with no holes, so any new point you pick decodes into a plausible example. A plain autoencoder only rebuilds inputs it was given.

Q. What are the two networks in a GAN, and what does each do?

The generator produces fakes meant to pass as real; the discriminator (a plain classifier) tries to tell fakes from genuine examples.

Q. In the GAN, what does the generator actually learn from?

Only the discriminator’s verdicts on its fakes. It never sees a real example directly; the realism comes from being pushed to fool a detector that keeps improving.

Q. How do a VAE's and a GAN's characteristic results tend to differ?

A VAE samples from an organized space and tends toward plausible but sometimes slightly blurry results. A GAN learns through a contest and tends toward sharp, convincing results, though its training is famously finicky to balance.

Q. Why is 'imagine' a misleading word for what a generative model does?

It learned the statistical shape of its training data and samples from it. It produces patterns like the ones it was trained on, rather than conjuring from understanding.