Start simple: the "average of averages" surprise
Roll one die. Every face — 1 through 6 — is equally likely, so the result is totally flat, no favourite. Now roll five dice and write down their average. Suddenly the middle values near 3.5 show up far more often than a 1 or a 6, because getting five low rolls (or five high ones) at once is rare. Average ten dice and the averages pile up into a smooth hill. That pile-up — flat input turning into a bell — is the Central Limit Theorem.
The thing you sample from (the dice, people's heights, order times) is the parent distribution; it has a mean $\mu$ and a spread $\sigma$. Grab $n$ values, average them, and you get one sample mean $\bar{x}$. Two facts make the magic: the sample means centre right on $\mu$, and they scatter less than single values — their spread is the standard error $SE=\sigma/\sqrt{n}$. The simplest form of the theorem is just $\bar{x}\approx\Normal{\mu}{\sigma/\sqrt{n}}$. Worked number: adult heights with $\mu=170$ cm and $\sigma=10$ cm. One person might be 150 or 190 cm. But average $n=25$ people and $SE=10/\sqrt{25}=2$ cm — so that average almost always lands within about 4 cm of 170.
For independent, identically distributed $X_i$ with finite mean $\mu$ and finite variance $\sigma^2$, the standardized mean
$$Z_n=\frac{\bar{X}_n-\mu}{\sigma/\sqrt{n}}\;\overset{d}{\longrightarrow}\;\Normal{0}{1}\quad\text{as }n\to\infty,$$
no matter what shape the parent has — equivalently $\bar{X}_n\approx\Normal{\mu}{\sigma^2/n}$. The only real requirement is finite variance (so the heavy-tailed Cauchy never settles down). How fast the bell appears depends on the parent's skewness; the Berry-Esseen bound below pins the error at order $1/\sqrt{n}$. In the sim, the Parent selector sets the shape of $X$, the n slider sets how many values go into each average, the +N buttons pile new sample means into the orange histogram, and the green curve is the target $\Normal{\mu}{\sigma^2/n}$.
Pick Exponential (very lopsided) with $n=1$ and play — the histogram copies the skewed parent. Now drag $n$ up to 30 and watch it straighten into the bell. Switch to Bernoulli (only 0s and 1s!) at $n=2$, then $n=50$ — pure discreteness smooths into a continuous curve. Finally open the vs n tab: since $SE=\sigma/\sqrt{n}$, quadrupling $n$ only halves the width, so the curve gets narrower and taller but never moves off $\mu$.
Central Limit Theorem — Laplace 1812 / Lindeberg-Lévy 1922
$$\boxed{\sqrt{n}\,\frac{\bar{X}_n - \mu}{\sigma} \;\overset{d}{\longrightarrow}\; \Normal{0}{1} \quad \text{as } n\to\infty}$$
Equivalently: $\bar{X}_n \approx \Normal{\mu}{\sigma^2/n}$ for large $n$, regardless of the parent distribution's shape.
| Symbol | Meaning | Type | Range | Interpretation |
|---|---|---|---|---|
| $X_i$ | Individual observations | RV | Depends on dist. | iid draws from parent distribution |
| $\mu$ | Population mean | Parameter | $(-\infty,\infty)$ | True mean of the parent distribution |
| $\sigma^2$ | Population variance | Parameter | $(0,\infty)$ | Must be finite for CLT to hold |
| $\bar{X}_n$ | Sample mean of n obs. | Statistic | $(-\infty,\infty)$ | Estimator of $\mu$; becomes more normal as $n$ grows |
| $\sigma/\sqrt{n}$ | Standard error of mean | Derived | $(0,\infty)$ | How much $\bar{X}_n$ varies; decreases as $1/\sqrt{n}$ |
| $\overset{d}{\to}$ | Convergence in distribution | Notation | — | CDF of LHS converges to CDF of RHS pointwise |
| $n$ | Sample size | Fixed | $\mathbb{N}$ | The "rule of thumb" is $n \ge 30$ (but not universal) |
Adding many independent random quantities — regardless of their individual distributions — produces a sum whose distribution converges to a bell curve. This happens because extreme values from different summands cancel out, while moderate values reinforce. The normal distribution is the unique "attractor" for normalized sums with finite variance.
Let $X_1,...,X_n \overset{iid}{\sim} F$ with $E[X]=\mu$ and $\text{Var}(X)=\sigma^2 < \infty$. Define the standardized mean:
$$Z_n = \frac{\bar{X}_n - \mu}{\sigma/\sqrt{n}} = \frac{\sum_{i=1}^n (X_i - \mu)}{\sigma\sqrt{n}}$$
We want to show $Z_n \overset{d}{\to} Z \sim \Normal{0}{1}$.
Let $Y_i = (X_i-\mu)/\sigma$ so $E[Y_i]=0$, $E[Y_i^2]=1$. Then $Z_n = \frac{1}{\sqrt{n}}\sum Y_i$. The MGF of $Z_n$ is:
$$M_{Z_n}(t) = E[e^{tZ_n}] = \left[M_Y\!\left(\frac{t}{\sqrt{n}}\right)\right]^n$$
by independence (MGF of sum = product of MGFs).
Expand $M_Y(s)$ around $s=0$ using $E[Y]=0$, $E[Y^2]=1$:
$$M_Y(s) = 1 + 0\cdot s + \frac{1}{2}s^2 + O(s^3) = 1 + \frac{s^2}{2} + O(s^3)$$
Substituting $s = t/\sqrt{n}$:
$$M_Y\!\left(\frac{t}{\sqrt{n}}\right) = 1 + \frac{t^2}{2n} + O(n^{-3/2})$$
$$M_{Z_n}(t) = \left(1 + \frac{t^2}{2n} + O(n^{-3/2})\right)^n \to e^{t^2/2} \quad \text{as } n\to\infty$$
using $\lim_{n\to\infty}(1 + a/n)^n = e^a$.
The MGF $e^{t^2/2}$ is exactly the MGF of $\Normal{0}{1}$. Since the MGF uniquely determines the distribution (under mild conditions), we conclude:
$$Z_n \overset{d}{\to} \Normal{0}{1}, \quad \text{i.e.,} \quad \bar{X}_n \overset{d}{\approx} \Normal{\mu}{\sigma^2/n}$$
The Lindeberg-Lévy CLT above requires iid. The more general Lindeberg CLT (1922) allows non-identical distributions, requiring only the Lindeberg condition: no single summand dominates the variance. The Lyapunov CLT adds a sufficient moment condition: $E[|Y_i|^{2+\delta}] < \infty$ for some $\delta > 0$.
How fast does the CLT kick in? The Berry-Esseen theorem quantifies this:
$$\sup_x |P(Z_n \le x) - \Phi(x)| \le \frac{C \cdot E[|X-\mu|^3]}{\sigma^3\sqrt{n}}, \qquad C \le 0.4748$$
The error decreases as $1/\sqrt{n}$. Symmetric distributions converge faster (skewness = 0 → lower $E[|Y|^3]$). The Exponential distribution (high skewness) requires larger $n$ for the same accuracy.
The Parent Distribution selector changes the shape from which individual $X_i$ are drawn. Switch to "Exponential" — extremely skewed (not bell-shaped). Yet watch what happens to the histogram of sample means $\bar{X}_n$ as you accumulate more samples — it converges to a normal bell curve regardless.
The n slider controls how many observations go into each sample mean. At $n=1$, the histogram of sample means mirrors the parent exactly. At $n=5$, it starts looking more bell-shaped. At $n=30$, it's nearly indistinguishable from normal for most parent distributions.
The green normal overlay curve is $\Normal{\mu}{\sigma^2/n}$ — the theoretical CLT approximation. Watch the orange histogram bars converge to this green curve as you accumulate more sample means. The convergence speed depends on the parent's skewness — Bimodal is fastest, Exponential is slowest.
Problem: A warehouse processes orders. Each order processing time follows Exponential(λ=0.5), so μ=2 min, σ²=4 min². If a day has n=100 orders, what is the probability the average processing time exceeds 2.3 minutes?
Step 1 — By CLT: $\bar{X}_{100} \approx \Normal{2}{4/100} = \Normal{2}{0.04}$, so $\sigma_{\bar{X}} = \sqrt{0.04} = 0.2$ min.
Step 2 — Standardize:
$$z = \frac{2.3 - 2}{0.2} = \frac{0.3}{0.2} = 1.50$$
Step 3 — Compute probability:
$$P(\bar{X} > 2.3) = P(Z > 1.50) = 1 - \Phi(1.50) = 1 - 0.9332 = \mathbf{0.0668}$$
Despite each individual order time being exponentially distributed (skewed!), the CLT lets us use the normal approximation for the average. A 6.7% chance of the day's average exceeding 2.3 min.
"The CLT says that with large n, the data itself becomes normally distributed."
The CLT says the sampling distribution of the sample mean $\bar{X}_n$ becomes approximately normal as $n \to \infty$. The individual observations $X_i$ retain their original distribution — if your data is exponential with $n=1000$ observations, those 1000 values are still exponentially distributed. But if you take 1000 different samples of size $n=30$ and compute 1000 sample means, those 1000 means follow approximately a normal distribution.
📖 Ross — A First Course in Probability, Ch. 8.3
"n ≥ 30 is always sufficient for the CLT approximation to be accurate."
The "n ≥ 30" rule of thumb is a rough guideline that works for mildly skewed distributions. For highly skewed distributions (like a Pareto with shape α close to 2) or heavy-tailed distributions, n=30 is far from sufficient. The Cauchy distribution has no mean and never satisfies the CLT — no amount of averaging makes it normal. The Berry-Esseen theorem shows convergence rate depends on the third absolute moment $E[|X|^3]$, which is large for highly non-normal parents.
📖 Casella & Berger — Statistical Inference, Ch. 5.5
"The CLT works because of some mysterious property of large samples; it's not rigorous."
The CLT is a rigorously proven mathematical theorem, not a heuristic. The Lindeberg-Lévy proof uses the moment generating function (or characteristic function) approach to show that the MGF of the standardized sum converges pointwise to $e^{t^2/2}$, the MGF of the standard normal. The convergence in distribution follows from Lévy's continuity theorem. The conditions (finite mean and variance) are clearly stated and necessary — the theorem fails for Cauchy (undefined mean) and Pareto with tail index ≤ 2 (infinite variance).
📖 DeGroot & Schervish — Probability and Statistics, Ch. 6.3
Using σ instead of σ/√n when computing probabilities about x̄
✅ The standard deviation of $\bar{X}_n$ is $\sigma/\sqrt{n}$ (the standard error), not $\sigma$. Using $\sigma$ gives probabilities about an individual observation, not about the mean. Example: with $\sigma=4, n=16$, the SE is $4/\sqrt{16}=1$, not 4.
🔍 Confusion between "distribution of X" and "distribution of X̄".
Applying the normal approximation to the sample mean when n is small AND the distribution is highly skewed
✅ Check both n and the parent distribution's skewness. For exponential data, prefer exact methods (gamma distribution for sum) or bootstrap when n<50. The simulation's Convergence tab shows how fast different parent distributions converge.
🔍 Students apply the "n≥30 rule" mechanically without checking parent distribution shape.
Confusing convergence in distribution with convergence in probability (Law of Large Numbers)
✅ LLN says $\bar{X}_n \overset{p}{\to} \mu$ (the mean converges to a constant). CLT describes the fluctuations around μ scaled by $\sqrt{n}$. Both are true simultaneously, but they describe different things: LLN tells us where the mean goes; CLT tells us the shape of the distribution of errors around μ.
🔍 Students conflate LLN ("sample mean → population mean") with CLT ("sampling distribution → normal").