Central Limit Theorem

It comes down to two structural facts. First, the normal is the unique distribution that is stable under sums in the right normalization: a sum of independent normals (after rescaling by $1/\sqrt n$) is again normal. Second, the normal maximizes entropy among all distributions with a given mean and variance — so when you average many independent contributions, the limiting distribution shouldn't add information beyond what's preserved (mean, variance), and entropy maximization picks the Gaussian. The proof reveals it: the Taylor expansion of any characteristic function up to second order is captured by the Gaussian's $e^{-t^2/2}$ — third and higher cumulants get washed out by the $1/\sqrt n$ scaling.

For each replication, it draws $n$ independent samples from the parent distribution using a deterministic Mulberry32 PRNG (so results are reproducible from the seed), computes their arithmetic mean $\bar X_n$, and stores it. After $M$ replications, you have $M$ values of $\bar X_n$, which it bins into a histogram and overlays the theoretical $\mathcal{N}(\mu,\sigma^2/n)$ density. The Kolmogorov–Smirnov readout is the supremum gap between the empirical CDF of those $M$ values and the predicted normal CDF — by CLT it shrinks at rate $1/\sqrt M$ for fixed $n$, plus a finite-$n$ bias that shrinks at rate $1/\sqrt n$ (Berry–Esseen).

It's the foundation of nearly all classical statistics. Confidence intervals: $\bar X \pm z_{\alpha/2}\sigma/\sqrt n$ assumes the CLT-normal sampling distribution. Hypothesis testing via $z$-tests and (with Slutsky's theorem) $t$-tests. A/B testing: companies decide whether button color matters by computing CLT-based $p$-values. Polling and surveys: margin of error $\propto 1/\sqrt n$ — that's why doubling sample size only halves uncertainty by $\sqrt 2$, not 2. Quality control: control charts use $\pm 3\sigma/\sqrt n$ Gaussian limits. Diffusion in physics: Brownian motion is the continuous limit of random walks — that's also CLT in disguise. Finance: option-pricing models assume log-returns are normal (debatable for tails, but CLT-based for averaged returns).

The classical CLT requires finite variance. Heavy-tailed distributions like Cauchy ($f(x)=1/(\pi(1+x^2))$) have $\E[X^2]=\infty$, so Step 3 of the proof breaks. Try Cauchy in the simulation: the histogram of sample means stays Cauchy-shaped and as wide as a single observation — averaging doesn't help. (In fact, $\bar X_n \sim \text{Cauchy}$ exactly for any $n$.) For such distributions, generalized CLT applies: properly normalized sums converge to $\alpha$-stable distributions, with $\alpha < 2$. Independence also matters: dependent sequences need mixing conditions or martingale CLTs. And "identically distributed" can be relaxed (Lindeberg–Feller) but you need a uniform bound on contributions.

The Berry–Esseen theorem quantifies it: $\sup_z \abs{\Pr(Z_n \le z) - \Phi(z)} \le \tfrac{C\,\rho}{\sigma^3 \sqrt n}$, where $\rho = \E[\abs{X_1 - \mu}^3]$ is the third absolute central moment and $C \approx 0.4748$ (current best constant). So the KS distance goes to zero at rate $1/\sqrt n$. For $n=30$ with skew-symmetric distributions, the bound is around $0.05$–$0.1$; for symmetric distributions like Uniform, the constant is much smaller in practice. The "rule of thumb" $n \ge 30$ is loosely tied to this — but for highly skewed parents (exponential, log-normal), $n=100$ may still show visible non-normality in the tails. The simulation's KS-vs-$n$ graph shows this $1/\sqrt n$ decay directly.

If you take a random walk on $\Z$ (each step $\pm 1$ with probability 1/2), the position after $n$ steps is a sum of i.i.d.\ Rademacher variables. By CLT, after $n$ steps the position $S_n \approx \mathcal{N}(0, n)$ — the spread grows as $\sqrt n$. Take the limit of small steps and small time intervals, scaling space by $\sqrt{\Delta t}$, and you get Brownian motion, whose density at time $t$ solves the heat equation $\partial_t u = \tfrac12 \partial_{xx} u$ with delta initial condition. The fundamental solution $\tfrac{1}{\sqrt{2\pi t}} e^{-x^2/(2t)}$ is exactly the CLT Gaussian with variance $t$. So CLT is the discrete-to-continuous bridge: random walks $\Rightarrow$ Brownian motion $\Rightarrow$ heat equation. The same Gaussian appears in Wiener integrals, stochastic differential equations, and Itô calculus.

No — CLT is about the distribution of the average, not the original variables. The $X_i$ stay exactly as they are: a Bernoulli is still 0 or 1. The sample mean $\bar X_n$ takes values in $\{0/n, 1/n, 2/n, \ldots, n/n\}$ — discrete, not continuous. But as $n$ grows, those discrete values get dense and the histogram of $\bar X_n$ looks like a continuous Gaussian density. The De Moivre–Laplace theorem (1733) is the special case of CLT for binomial — it predates the general CLT and is what gave us the Gaussian shape originally.