Line up everyone in your class by height. A few are very short, a few are very tall, but most cluster near the middle. If you stack people into height bins, the bars make a rounded hill — high in the center, sloping away on both sides. That hill shape, showing up everywhere from shoe sizes to exam scores to the tiny wobble in repeated measurements, is the normal distribution, the famous "bell curve."
Two numbers describe the hill completely. The middle is the average, written $\mu$ ("mew") — it's where the peak sits. The spread is the standard deviation $\sigma$ ("sigma") — how wide the hill is. Slide the whole curve left or right and you change $\mu$; squash it skinny or stretch it wide and you change $\sigma$. Nothing else is needed to pin down the shape.
Here is the rule worth memorizing: in any bell curve, about 68% of values land within one $\sigma$ of the mean, about 95% within two, and almost everything (99.7%) within three. Say adult women's heights average $\mu = 165$ cm with $\sigma = 7$ cm. Then roughly 68% of women fall between $165 - 7 = 158$ cm and $165 + 7 = 172$ cm. A height of 179 cm is two $\sigma$ above average, placing it in the top ~2.5% — tall, but not unheard of.
The exact curve is $f(x) = \frac{1}{\sigma\sqrt{2\pi}}\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right)$. The $(x-\mu)^2$ in the exponent makes it symmetric and decay fast; the $1/(\sigma\sqrt{2\pi})$ out front is exactly the amount needed so the total area equals 1. To compare any normal with any other, convert to a z-score $z = \frac{x-\mu}{\sigma}$ — literally "how many $\sigma$ from the mean" — and then one standard table $\Phi(z)$ gives every probability: $P(a \le X \le b) = \Phi\!\left(\frac{b-\mu}{\sigma}\right) - \Phi\!\left(\frac{a-\mu}{\sigma}\right)$. In the sim above the μ slider shifts the curve, the σ slider sets its width, and the a and b sliders fence off the shaded probability between them.
Drag σ down to 0.5 and watch the bell turn into a tall spike — small spread means values huddle near $\mu$. Set a $= \mu-\sigma$ and b $= \mu+\sigma$ and read the shaded probability: it settles near 0.68, the rule in action. Then click +1000 a few times and watch the orange sample histogram fall onto the smooth violet curve — that is what "the PDF describes where data lands" looks like in motion.
Normal (Gaussian) Distribution — Gauss 1809 / de Moivre 1733
$$X \sim \Normal{\mu}{\sigma^2}$$
$$\boxed{f(x;\mu,\sigma) = \frac{1}{\sigma\sqrt{2\pi}}\,\exp\!\left(-\frac{(x-\mu)^2}{2\sigma^2}\right), \quad x\in\mathbb{R}}$$
| Symbol | Meaning | Type | Range | Interpretation |
|---|---|---|---|---|
| $\mu$ | Population mean | Parameter | $(-\infty,\infty)$ | Center of the bell curve; location parameter |
| $\sigma$ | Population std dev | Parameter | $(0,\infty)$ | Controls width — larger $\sigma$ = flatter curve |
| $\sigma^2$ | Population variance | Parameter | $[0,\infty)$ | Average squared deviation from $\mu$ |
| $f(x)$ | Probability density | Function | $[0,\infty)$ | Height of curve; $f(x)\,dx \approx P(x \le X \le x+dx)$ |
| $\Phi(z)$ | Standard normal CDF | Function | $(0,1)$ | $P(Z \le z)$ where $Z \sim \Normal{0}{1}$ |
| $z$ | Standard score (z-score) | Derived | $(-\infty,\infty)$ | Signed distance from $\mu$ in units of $\sigma$ |
| $\bar{x}$ | Sample mean | Statistic | $(-\infty,\infty)$ | Estimator of $\mu$ from $n$ observations |
| $s^2$ | Sample variance | Statistic | $[0,\infty)$ | Unbiased estimator of $\sigma^2$ |
We want a continuous distribution symmetric around $\mu$ where values near $\mu$ are most probable and probability decays quickly away from $\mu$. The simplest function with this property is $f(x) \propto e^{-c(x-\mu)^2}$ for some $c > 0$.
Choosing $c = \frac{1}{2\sigma^2}$ ensures the variance equals $\sigma^2$. So our unnormalized density is $g(x) = e^{-(x-\mu)^2/(2\sigma^2)}$.
We need $\int_{-\infty}^{\infty} g(x)\,dx = 1$. Using the Gaussian integral result $\int_{-\infty}^{\infty} e^{-t^2}\,dt = \sqrt{\pi}$ with substitution $t = (x-\mu)/(\sigma\sqrt{2})$:
$$\int_{-\infty}^{\infty} e^{-(x-\mu)^2/(2\sigma^2)}\,dx = \sigma\sqrt{2\pi}$$
Therefore the normalizing constant is $\frac{1}{\sigma\sqrt{2\pi}}$.
$$E[X] = \int_{-\infty}^{\infty} x\cdot f(x)\,dx$$
Substituting $u = x - \mu$, odd integrand vanishes: $E[X] = \mu\int f\,dx + \int u\cdot f\,du = \mu\cdot 1 + 0 = \mu$.
$$\text{Var}(X) = E[(X-\mu)^2] = \int_{-\infty}^{\infty} (x-\mu)^2 f(x)\,dx$$
With $u=(x-\mu)/\sigma$, this becomes $\sigma^2 \int_{-\infty}^{\infty} u^2\,\frac{e^{-u^2/2}}{\sqrt{2\pi}}\,du$. Integration by parts gives $= \sigma^2\cdot 1 = \sigma^2$.
If $X\sim\Normal{\mu}{\sigma^2}$ then $Z = \frac{X-\mu}{\sigma}\sim\Normal{0}{1}$. This lets us compute all probabilities from a single standard table:
$$P(a \le X \le b) = \Phi\!\left(\frac{b-\mu}{\sigma}\right) - \Phi\!\left(\frac{a-\mu}{\sigma}\right)$$
$$P(\mu-\sigma \le X \le \mu+\sigma) = 2\Phi(1)-1 \approx 0.6827$$
$$P(\mu-2\sigma \le X \le \mu+2\sigma) \approx 0.9545$$
$$P(\mu-3\sigma \le X \le \mu+3\sigma) \approx 0.9973$$
The standard normal CDF has no closed form, but is expressed via the error function:
$$\Phi(z) = \frac{1}{2}\left[1 + \text{erf}\!\left(\frac{z}{\sqrt{2}}\right)\right], \quad \text{erf}(x)=\frac{2}{\sqrt{\pi}}\int_0^x e^{-t^2}\,dt$$
The MGF of $X\sim\Normal{\mu}{\sigma^2}$ is:
$$M_X(t) = E[e^{tX}] = \exp\!\left(\mu t + \frac{\sigma^2 t^2}{2}\right)$$
This uniquely characterizes the normal distribution and is used to prove the CLT via characteristic functions.
The μ slider is the population mean $\mu$ — it shifts the entire bell curve left or right along the x-axis without changing its shape. Samples drawn cluster around this value.
The σ slider is the population standard deviation $\sigma$ — increasing it flattens and widens the curve (more uncertainty); decreasing it creates a tall, narrow spike (high concentration near $\mu$). Watch how the histogram mirrors this shape as $n$ grows.
The a and b sliders set the integration bounds — the purple shaded area on the PDF is exactly $P(a \le X \le b) = \Phi\!\left(\frac{b-\mu}{\sigma}\right) - \Phi\!\left(\frac{a-\mu}{\sigma}\right)$.
As you click Draw 1 Sample repeatedly, watch $\bar{x}$ converge toward $\mu$ — this is the Law of Large Numbers in action. The orange bars (histogram) converge to the violet curve (theoretical PDF) — this is what "the PDF describes where samples fall" means visually.
Problem: Heights of adult women follow $X\sim\Normal{165}{7^2}$ cm. Find $P(158 \le X \le 172)$.
Step 1: Standardize the bounds:
$$z_1 = \frac{158-165}{7} = \frac{-7}{7} = -1.00, \quad z_2 = \frac{172-165}{7} = \frac{7}{7} = 1.00$$
Step 2: Look up (or compute) $\Phi(1.00) = 0.8413$ and $\Phi(-1.00) = 1 - 0.8413 = 0.1587$.
Step 3: Compute probability:
$$P(158 \le X \le 172) = \Phi(1.00) - \Phi(-1.00) = 0.8413 - 0.1587 = \mathbf{0.6827}$$
This is the familiar 68.27% rule — approximately 68% of women have heights within one standard deviation of the mean. Try setting μ=165, σ=7, a=158, b=172 in the simulation to verify this!
"The normal distribution is the most common distribution in nature — all real data is normally distributed."
Many real-world distributions are skewed, heavy-tailed, multimodal, or strictly non-negative. Income distributions are right-skewed (Pareto/log-normal). Survival times follow exponential or Weibull distributions. Counts follow Poisson distributions. The normal distribution is a mathematical model that is sometimes a good approximation for averages (via CLT) — not a universal law of nature. Normality is an assumption to be tested, not assumed.
📖 Freedman, Pisani & Purves — Statistics, Ch. 5: "The Normal Approximation for Data"
"$P(X = 1.5) = f(1.5)$ — the PDF value is the probability at that point."
For any continuous random variable, $P(X = c) = 0$ exactly for every specific value $c$ — this is because a single point has measure zero on the real line. The PDF $f(x)$ is a density, not a probability. Only integrals over intervals give probabilities: $P(a \le X \le b) = \int_a^b f(x)\,dx$. Also note $f(x)$ can exceed 1 for concentrated distributions (e.g. $\Normal{0}{0.1^2}$ has peak $\approx 3.99$).
📖 Ross — A First Course in Probability, Ch. 5.1
"A larger σ means the distribution is 'worse' or 'less accurate'."
Standard deviation $\sigma$ is a property of the population — it describes natural variability, not error. Heights having $\sigma = 7$ cm is simply a fact about human variation, not an inaccuracy. Larger $\sigma$ means more spread in the phenomenon itself. What matters for estimation precision is the standard error of the mean: $\text{SE} = \sigma/\sqrt{n}$, which decreases as sample size $n$ increases.
📖 DeGroot & Schervish — Probability and Statistics, Ch. 4.3
Wrong: $z = \frac{\mu - x}{\sigma}$ (subtracting in wrong order)
✅ Always compute $z = \frac{x - \mu}{\sigma}$. The sign matters: positive $z$ means $x$ is above the mean. Reversing gives the wrong tail.
🔍 Why: Students memorize the formula without internalizing that $x$ is the observation and $\mu$ is the reference.
Wrong: Using $\sigma$ when you should use $\sigma/\sqrt{n}$ for the sampling distribution
✅ When computing probabilities about the sample mean $\bar{X}$, the relevant standard deviation is the standard error $\sigma/\sqrt{n}$, not $\sigma$. Individual observations spread with $\sigma$; sample means spread with $\sigma/\sqrt{n}$.
🔍 Why: Confusion between the distribution of $X$ and the distribution of $\bar{X}$.
Wrong: $P(X \ge z) = \Phi(z)$ instead of $1 - \Phi(z)$
✅ $\Phi(z) = P(Z \le z)$ is the left-tail probability. For the right tail: $P(Z \ge z) = 1 - \Phi(z)$. For a two-tailed interval: $P(|Z| \le z) = 2\Phi(z) - 1$.
🔍 Why: Students forget which tail the standard table gives.