Multinomial Distribution

Explore joint category counts in fixed independent trials, exact ordering probabilities, binomial marginals and negative count covariance.

Multinomial Distribution

Concept Overview

Repeat an experiment a fixed number of times. Each trial independently produces exactly one category, with the same category probabilities on every trial. The multinomial distribution describes the joint totals. This explorer uses three categories A, B and C. Its count vectors occupy a triangle because A + B + C must equal the trial count.

Mathematical Definition

For n trials and category probabilities p_A, p_B, p_C summing to one, let X_A, X_B, X_C count the outcomes. For nonnegative integers a, b, c with a + b + c = n:

P(X_A=a, X_B=b, X_C=c) = n!/(a! b! c!) · p_A^a p_B^b p_C^c

c = n − a − b; 0! = 1

p_i = w_i / (w_A + w_B + w_C)

Any one ordering with these counts has the product probability. The coefficient n!/(a! b! c!) counts distinct orderings, so multiplying it by the product gives the count event probability. If a + b exceeds n, no nonnegative c completes the vector and the event is impossible. There are (n + 1)(n + 2)/2 structurally feasible three-category vectors. Some have zero probability when a category has probability zero.

Key Concepts

A counted example

With six trials and weights (1, 2, 3), the probabilities are (1/6, 1/3, 1/2). For counts (1, 2, 3), there are 6!/(1! 2! 3!) = 60 orderings. Each ordering has probability (1/6)(1/3)²(1/2)³ = 1/432. Thus the joint event has probability 60/432 = 5/36. The table includes all 28 count vectors, whose joint probabilities add to one. They are not equally likely.

Binomial marginals

To study A alone, group B and C into “not A.” Each trial becomes a Bernoulli experiment with probability p_A. Summing joint table entries with the same A count therefore gives a binomial PMF. The same reasoning applies to B and C.

X_i ~ Binomial(n, p_i)

P(X_i=k) = C(n,k) p_i^k(1 − p_i)^n−k

E[X_i] = np_i; Var(X_i) = np_i(1 − p_i)

Independent trials, dependent counts

Within one trial, two different categories cannot both occur. Their indicator product is zero, and their covariance is −p_ip_j. Indicators from different trials are independent. Summing across n trials gives:

Cov(X_i, X_j) = −np_ip_j, i ≠ j

The covariance is negative when n and both probabilities are positive, and zero in degenerate cases. An extra A count leaves fewer trials for B or C. The covariance matrix is singular because the sum of the counts is fixed: the sum of each matrix row is zero. Multiplying the three binomial marginal probabilities would incorrectly assume the counts are independent.

Boundaries and model assumptions

With zero trials the only count vector is (0, 0, 0), with probability one and zero covariance. A category of probability zero cannot have a positive count; its zero-count factor is the empty product, equal to one. If only one weight is positive, all trials land in that category. All-zero weights cannot define probabilities and are rejected even when n = 0. The explorer supports integer n from 0 to 30 and weights from 0 to 100. It preserves impossible query vectors explicitly and rejects blank, fractional or out-of-range entries. Table values round to three decimals, so small positive cells may display as zero. Primary probability outputs and tooltips use scientific notation when a nonzero value falls below six-decimal resolution. Trials must be independent with fixed probabilities: finite sampling without replacement generally needs a hypergeometric model.

Historical Context

The counting formula is connected to the multinomial theorem. Expanding (p_A + p_B + p_C)ⁿ groups terms by how often each category is chosen. The resulting coefficients count orderings, and the expansion explains why all joint probabilities sum to one.

Real-world Applications

Repeated categorical experiments: Count three classes of die outcomes in a fixed batch of independent rolls.
Survey categories: Model category totals under an independent sampling assumption with fixed response probabilities.
Classification counts: Compare observed class frequencies to a specified categorical model, while checking its sampling assumptions.

Related Concepts

Alias Method Sampling — efficiently draw a single categorical outcome from weights.
Entropy & Information Theory — measure uncertainty in the probabilities on one trial.
Hypergeometric Distribution — finite sampling without replacement changes the dependence between trials.

References

SciPy: multinomial — PMF, count constraints and zero-probability endpoints.
Kyle Siegrist, Random: The Multinomial Distribution — counting derivation, marginals, moments and covariance.
NIST Dataplot: Multinomial PDF — fixed category probabilities and integer count vectors.

Multinomial Distribution