Updating a hunch with evidence
Suppose a friend hands you a bent coin and asks, "what's the chance it lands heads?" Even before the first flip you have a hunch — probably near a half, but you're not sure. Flip it ten times, see seven heads, and your hunch slides toward "this coin favours heads." Bayesian updating is exactly that move, made precise: start with a belief, watch some data, end with a sharper belief.
We store the starting belief as a Beta distribution with two dials, $\alpha_0$ and $\beta_0$. Read them as pretend tallies you have already seen: $\alpha_0$ imagined heads and $\beta_0$ imagined tails. A flat "I have no idea" start is $\text{Beta}(1,1)$. Now collect real data — $k$ heads out of $n$ flips — and the update could not be simpler: just add the real counts to the pretend ones.
$$\text{posterior}=\text{Beta}(\alpha_0+k,\ \beta_0+n-k)$$
Start at $\text{Beta}(2,2)$, observe $k=7$ heads in $n=10$ flips, and the belief becomes $\text{Beta}(9,5)$, centred at $9/14\approx0.64$.
That centre is the posterior mean $\hat p=\dfrac{\alpha_0+k}{\alpha_0+\beta_0+n}$, a weighted average of the prior mean and the data's own estimate $k/n$. The prior carries weight $\alpha_0+\beta_0$ — its effective "pseudo-sample size" — and the data carries weight $n$, so a strong prior pulls the answer toward your starting hunch while a flood of data washes the prior out. This clean "add the counts" rule works only because Beta is the conjugate prior for a binomial likelihood: prior and posterior share the same shape, so no simulation is needed. On the controls, the alpha and beta sliders set $\alpha_0,\beta_0$, while $k$ and $n$ set the data; the shaded band is the 95% credible interval — the range that holds $p$ with 95% probability given what you have seen.
Set the prior to $\text{Beta}(10,10)$ (strong) and watch the posterior barely budge toward the data; then drop to $\text{Beta}(1,1)$ and watch the data take over. Hold the prior fixed and raise $n$ while keeping the same heads ratio — the posterior peak narrows as certainty grows. Finally, open the Sequential Update tab and click "+10 Flips" a few times to watch the posterior mean home in on the true $p$ line.
Beta-Binomial Conjugate — Thomas Bayes 1763 / Laplace
Prior: $p\sim\text{Beta}(\alpha_0,\beta_0)$. Likelihood: $k\mid p\sim\text{Bin}(n,p)$.
$$\boxed{\text{Posterior: }p\mid k,n\sim\text{Beta}(\alpha_0+k,\;\beta_0+n-k)}$$
Posterior mean: $\hat p=\dfrac{\alpha_0+k}{\alpha_0+\beta_0+n}$
| Prior | Data: k=7, n=10 | Posterior | Post. mean |
|---|---|---|---|
| Beta(1,1) — Uniform | 7 heads / 10 flips | Beta(8,4) | 8/12=0.667 |
| Beta(2,2) — Weak symmetric | 7 heads / 10 flips | Beta(9,5) | 9/14=0.643 |
| Beta(10,10) — Strong symmetric | 7 heads / 10 flips | Beta(17,13) | 17/30=0.567 |
| Beta(2,8) — Biased toward 0.2 | 7 heads / 10 flips | Beta(9,11) | 9/20=0.450 |
Beta is conjugate to Binomial: prior × likelihood = same distributional form. This means the posterior is analytically tractable — no MCMC needed! The prior alpha and beta play the role of "pseudo-counts": alpha represents prior successes, beta represents prior failures. Prior strength = alpha+beta (effective prior sample size).
95% Credible Interval (CrI): P(p ∈ CrI | data) = 0.95. The probability that p is inside the interval IS 0.95. This is the direct interpretation people often wrongly assign to CI. 95% CI: if we repeated the experiment many times, 95% of constructed CIs would contain the true p — but a specific CI either contains p or doesn't.
"A 95% CI means there's a 95% chance the parameter is inside."
The frequentist CI is a PROCEDURE that captures the true p 95% of the time over repeated experiments. The specific interval [0.4, 0.8] either contains p (probability 1) or doesn't (probability 0) — p is fixed, not random. The Bayesian Credible Interval directly gives P(p ∈ CrI | data) = 0.95 — this IS the probability the parameter is inside, treating p as random.
📖 Gelman et al. — BDA Ch. 2
"The Beta(1,1) uniform prior is always 'non-informative'."
Beta(1,1) is flat on [0,1] but NOT invariant under reparametrization. If you use phi=log(p/(1-p)) (log-odds), the induced prior on phi is NOT uniform. The Jeffreys prior (Beta(0.5,0.5)) IS invariant under reparametrization — it's the "truly non-informative" prior for Bernoulli. In practice, prior sensitivity analysis (Prior Sensitivity tab) should always be performed.
📖 Gelman et al. — BDA Ch. 2.4
Forgetting that Bayesian posteriors are distributions, not point estimates
✅ The posterior mean, mode (MAP), and median are all valid summaries, but the full posterior distribution contains all the information. Always report the full posterior or at least the credible interval. For decision-making, use the full posterior with a loss function — different loss functions give different optimal decisions.
🔍 Reporting only MAP estimate without uncertainty interval.