Updating beliefs with evidence: Bayes' theorem

What you’ll learn

This is lesson 7 of Track 9 (Statistics & Probability for AI) and the close of Phase 2 (The laws of chance). The previous lesson ended on a cliffhanger: the chance of A given B is not the chance of B given A, and we needed a way to convert one into the other. That way is Bayes’ theorem, the mathematics of changing your mind correctly. You will learn to update a belief when evidence arrives, keeping the base rate in the calculation so a strong-sounding result does not run away with your conclusion. The source curriculum is Khan Academy’s Statistics & Probability course, by Sal Khan and the Khan Academy team, freely available and cited as further study.

The lesson builds Bayes intuitively first, from natural frequencies (counting a concrete population), then writes the formula and names its four parts. It re-derives lesson 1’s base-rate result exactly (a 99%-accurate test for a 1-in-100 disease, still only 50% on a positive), shows how a second independent positive updates the belief to 99%, and closes on where Bayes lives in AI: spam filtering, combining a detector’s output with the base rate, and the discipline of updating as data arrives.

Where this fits

This is lesson 7 of 14 and the final lesson of Phase 2. It is the payoff of an arc that began in lesson 1 (the base-rate preview) and was set up in lesson 6 (the two different conditionals on a two-way table). The next lesson, Random variables and expected value, opens Phase 3 (Random variables and the distributions that matter), turning from single events to whole distributions of outcomes.

Before you start

Prerequisites: the previous lesson (Conditional probability and independence), since Bayes is built directly on conditional probability and the two-way table. Lesson 1’s base-rate example is the motivating story. Comfort with fractions and multiplying a few decimals is all the math you need.

About the math

This lesson has a real formula, but it is introduced gently and always shadowed by the natural-frequencies version (just counting a population), which gives the same answer with less symbol-pushing. The arithmetic is multiplying two decimals and dividing, all worked step by step. If you can follow “0.99 times 0.01,” you can follow every calculation here.

By the end, you’ll be able to

State Bayes’ theorem and name its parts (prior, likelihood, evidence, posterior)
Use natural frequencies to compute a posterior probability without the formula
Re-derive the base-rate result from lesson 1 and explain why a positive test can still mean a low probability
Update a belief twice, using the first posterior as the second prior
Recognize Bayesian updating in AI (spam filtering, combining a base rate with new evidence) and name base-rate neglect

Time and difficulty

Read time: about 12 minutes
Practice time: about 16 minutes (a self-check, a natural-frequencies security-alert computation, a name-the-parts-and-predict exercise, and flashcards)
Difficulty: standard (one formula, always backed by counting; arithmetic is multiply-and-divide)