Probability & Statistics

Bayes' Theorem

A fundamental theorem for updating probabilities based on new evidence.

By Riano · Published · Updated · Report a correction

Bayes' theorem: count every way a signal can appear

A signal can be common in one group without most signals coming from that group. This experiment separates those two questions. Calculate the numbers of marked and unmarked tokens that trigger a toy sensor, then use Riano's three probability sliders to see how the posterior changes. The rates below are assumptions for learning, not measurements of a real sensor.

Which conditional probability are you asking for?

Let A mean “the token is marked” and B mean “the sensor signals.” The prior P(A) describes the population before the signal. The true-positive rate P(B|A) describes how the sensor behaves on marked tokens. We want P(A|B), the proportion of marked tokens among all tokens that signal. Reversing the order changes the population in the denominator.

P(A|B) = P(B|A)P(A) / [P(B|A)P(A) + P(B|not A)P(not A)]

This form assumes the two groups cover the population and the denominator P(B) is positive. The false-positive rate is P(B|not A); specificity would be one minus that rate.

Worked example: a 90% true-positive rate gives a 50% posterior

Suppose 10% of tokens are marked, the sensor signals for 90% of marked tokens, and it also signals for 10% of unmarked tokens. Use a population of 1,000 tokens with exactly these proportions to make the denominator visible.

  1. There are 100 marked tokens. Of those, 90 signal and 10 do not.
  2. There are 900 unmarked tokens. Of those, 90 signal and 810 do not.
  3. There are 180 signals altogether, including 90 marked tokens. The posterior is 90/180 = 0.5, or 50%.

The evidence probability is P(B) = 0.9 × 0.1 + 0.1 × 0.9 = 0.18. Dividing only by the marked population would answer a different question. This frequency construction illustrates the probabilities; a random sample of 1,000 is not guaranteed to contain exactly these counts.

Change one assumption and predict the result
PriorTrue-positive rateFalse-positive rateEvidencePosterior
10%90%10%18.0%50.0%
1%90%10%10.8%8.3%
10%90%1%9.9%90.9%

Reproduce the calculation in the module

Open the Bayes' Theorem experiment. Its sliders describe a prior, sensitivity and false-positive rate. Keep track of which group each rate conditions on.

  1. Set the prior to 0.10, sensitivity to 0.90 and false-positive rate to 0.10. Check the 18.0% evidence probability and 50.0% posterior.
  2. Change only the prior to 0.01. There are now 9 marked signals and 99 unmarked signals per 1,000 model tokens: 9/108 is about 8.3%.
  3. Restore the prior to 0.10 and reduce the false-positive rate to 0.01. The ratio becomes 90/(90 + 9), approximately 90.9%.

The grid represents individual categories using a finite number of squares. Its counts are rounded, while the numeric probabilities come from the formula. At other settings, counting squares can differ slightly from the displayed probability.

What the diagram cannot establish

The calculation depends on the supplied population rate and both sensor rates. A visualization cannot establish that those assumptions describe a real population, and a posterior is not evidence that the sensor caused the marked condition. If P(B) = 0, the elementary conditional-probability ratio is undefined; a software fallback must not be interpreted as a valid conditional probability. Some exact endpoints are outside the module's slider ranges.

Repeatedly multiplying likelihoods also requires assumptions about how observations depend on one another. Two correlated sensor readings do not automatically supply two independent pieces of evidence.

Predict before moving the slider

With sensitivity 90% and false-positive rate 10%, what prior makes half the signals false positives? Explain it using group counts. Then set the prior to 50% and calculate the new posterior before looking at the screen.

Check the reasoning

The original 10% prior gives 90 marked and 90 unmarked signals. A 50% prior instead gives 450 marked and 50 unmarked signals, so the posterior is 90%. Matching the sensitivity in this second case is a consequence of these particular rates, not a general identity.

For a longer sequence of predictions, use the worked Bayesian updating guide.

Sources for the probability rules

MIT OpenCourseWare, Reading 3: Conditional Probability, Independence and Bayes' Theorem develops conditioning and Bayes' rule. The token counts and slider comparisons here are Riano's worked examples applying those rules.

Experience it interactively

Adjust parameters, observe in real time, and build deep intuition with Riano’s interactive Bayes' Theorem module.

Try Bayes' Theorem on Riano →

More in Probability & Statistics