Dirichlet Distribution

Dirichlet Distribution

Mathematical Definition

Let α = (α_A, α_B, α_C) have positive entries, and let S be their sum. With p_C = 1 − p_A − p_B, the density with respect to the two free coordinates dp_A dp_B is:

The normalizing factor makes the integral over the simplex equal to one. It is the reciprocal multivariate beta function. For positive integer parameters, Γ(m) = (m − 1)!, making every example in this explorer independently checkable with factorials.

Key Concepts

Density is not the probability of a point

A density may exceed one. The probability of a region is its density integral, and the probability of one exact probability vector is zero. For α = (1, 1, 1), the density is constant at 2: the region 0 ≤ p_A, 0 ≤ p_B, p_A + p_B ≤ 1 has coordinate area 1/2, so 2 × 1/2 = 1. The drawn triangle is an affine barycentric display; its screen area is not the measure used by the formula. Color samples density rather than assigning probability mass to pixels.

Mean, concentration and dependence

Scaling all α entries by the same positive factor preserves the mean. Increasing S at that fixed mean reduces the variance. An update may also change the mean, so concentration alone does not describe every change of shape. The probability components have negative covariance: increasing one leaves less of their fixed total for the others. Every covariance row sums to zero; the matrix is singular because the component sum is constant.

A single component has the marginal distribution Beta(α_i, S − α_i). The three components are not independent, so multiplying their marginal densities does not give the joint density.

Conjugate updating from counts

Suppose trials are independent conditional on one fixed unknown probability vector, and their observed counts are x_A, x_B, x_C. The multinomial likelihood contributes powers p_i^xi. Multiplying it by the prior adds these exponents. After normalization, the posterior is another Dirichlet distribution:

For n > 0 the posterior mean is a weighted average of the prior mean and observed frequency: weights S/(S + n) and n/(S + n). With no observations the posterior equals the prior; there is no observed frequency to divide by n. The predictive statement concerns one next category. Conditional independence of trials does not make probability components independent.

A worked update

Start with α = (2, 2, 2), and observe counts (3, 1, 2). The posterior parameters are (5, 3, 4), with concentration 12. The posterior means are (5/12, 1/4, 1/3), while the observed frequencies are (1/2, 1/6, 1/3). At p = (0.30, 0.30, 0.40), the prior density is 120 × 0.30 × 0.30 × 0.40 = 4.32. The posterior density is 138600 × 0.30⁴ × 0.30² × 0.40³ = 6.4665216. These are densities, not event probabilities.

Modes and finite boundaries

For α_i ≥ 1 and S > 3, the joint mode is p_i = (α_i − 1)/(S − 3); an α entry of 1 can place it on an edge or vertex. When all entries are 1, the distribution is uniform and has no unique mode. The mode generally differs from the mean.

This bounded explorer uses integer prior entries 1–8 and observed counts 0–20 per category. Its density is a polynomial, so it has a finite continuous extension to the closed simplex: a zero coordinate with exponent zero contributes 1, while a positive exponent contributes zero. For α = (2, 1, 1), f(p) = 6p_A, reaching 6 at the A vertex. A boundary density is still not a point mass. The full Dirichlet family permits positive entries below 1 and singular boundary densities; those shapes are outside this UI.

Reading and operating the diagram

The prior mean is a ring, the posterior mean a green cross, and the white point is the query. Drag that point to query whole-percent coordinates, or edit A and B percentages directly; C fills the remaining budget. Background drag, arrow keys and Home navigate the diagram. Reset view changes navigation only; Reset example restores all model fields. Color uses the square root of sampled density divided by the selected view’s peak, so compare numeric densities when switching views. The mesh is a visual approximation; reported moments and query densities come from analytic formulas. Outputs round to six decimals and preserve tiny nonzero densities with scientific notation. Invalid inputs remove prior results instead of substituting a distribution.

Historical Context

The Dirichlet family connects the classical multivariate beta integral with probability distributions on proportions. Conjugate Bayesian analysis uses its power-product form to turn categorical observations into parameter additions. Its relationship with reinforced urn models gives another way to understand why accumulated counts influence prediction.

Real-world Applications

• Categorical estimation: Combine a stated prior with category counts while retaining uncertainty over their probabilities.
• Mixture proportions: Express beliefs about the nonnegative weights of components that sum to one.
• Sequential prediction: Update a categorical model as new observations arrive and compute one-step predictive probabilities.

Related Concepts

• Beta Distribution — a two-category Dirichlet and the marginal law of each component.
• Multinomial Distribution — the conditional likelihood for observed category totals.
• Bayes’ Theorem — posterior density is proportional to prior density times likelihood.
• Pólya Urn Model — reinforcement relates to the integrated categorical prediction scheme.

References

• SciPy: Dirichlet distribution — concentration parameters, normalized density and simplex.
• University of Washington: Multinomial distribution — Dirichlet prior, conjugacy and posterior mean.
• Stanford Stats366: Modelling Mixtures — Dirichlet updating, mixture proportions and the urn connection.