Beta–Bernoulli Point-Null Bayes Factor

Compare a point null with a continuous alternative using integrated evidence and model odds.

Beta–Bernoulli Point-Null Bayes Factor

In Bayesian inference, a Bayes factor compares the predictive success of two models. Here we compare a point null hypothesis (H₀: p = 0.5) against a continuous alternative hypothesis (H₁: p ~ Beta(a, b)) for Bernoulli sequence data.

Worked example

Consider observing exactly 2 successes in 4 trials. Let's calculate the Bayes factor for the alternative H₁ (Beta(1,1)) over the null H₀ (p=0.5).

Null Evidence (m₀): Under H₀, p is fixed at 0.5. The probability of any specific ordered sequence of 4 trials is (0.5)⁴ = 0.0625.
Alternative Evidence (m₁): Under H₁, p ~ Beta(1,1) (a uniform prior). The marginal likelihood of observing 2 successes and 2 failures is:
(1 * 2) * (1 * 2) / (2 * 3 * 4 * 5) = 4 / 120 = 0.0333...
Bayes Factor (BF): The ratio m₁ / m₀ is 0.0333... / 0.0625 = 0.5333...
Posterior Probability: If the prior probability of H₁ is 0.5, the posterior probability of H₁ becomes 0.5333... / (1 + 0.5333...) = 0.3478...

Experiment

Open the Beta–Bernoulli Point-Null Bayes Factor module to explore this interactively.

Set Trials to 4 and Successes to 2.
Set Beta alpha and Beta beta to 1 (Uniform prior).
Observe the Bayes factor alternative over null of approximately 0.5333.
Set Successes to 4. Notice the Bayes factor increases to 3.2, favoring the alternative, and the posterior probability of H₁ increases to 0.7619...
Change the Prior alternative to 0 and observe the posterior alternative probability immediately drop to 0, regardless of the Bayes Factor.

Limitations

The Bayes factor strictly depends on the choice of the prior for the alternative hypothesis. A very diffuse prior (like a Beta with very small parameters) might heavily penalize the alternative model, an effect related to Lindley's paradox. Furthermore, calculating marginal likelihoods analytically is only possible for conjugate priors like the Beta-Bernoulli pair; more complex models require numerical integration or MCMC approximations. Note that the evidence probabilities shown are for the specific ordered sequence of trials; multiplying both evidences by the binomial coefficient would cancel out in the Bayes factor calculation.

References

Helsinki Data Science Course: Inference covers Bayesian inference and model comparison using Bayes factors.

Beta–Bernoulli Point-Null Bayes Factor