Flip a coin 100 times to learn how biased it is. Do you read your friend all 100 results — H, T, T, H, …? Of course not. You just say "57 heads." That single count tells them everything about the coin's bias; the order the flips came in never mattered. That number is a sufficient statistic — a summary that throws away nothing important about the thing you're trying to learn.
Call the measurements $X_1,X_2,\dots,X_n$ and the quantity you want to learn $\theta$ (the coin's true heads-probability, or a population average). A statistic $T$ is just some number computed from the data. We say $T$ is sufficient for $\theta$ when, once you know $T$, the leftover details of the data tell you nothing more about $\theta$. For coin flips — and a surprising number of other cases — that magic summary is simply the total, $T=\sum_i X_i$. With $n=100$ flips and $T=57$ heads, the best estimate is $\hat p = T/n = 0.57$. You never needed the individual flips at all.
The Factorization Theorem makes "sufficient" exact: $T$ is sufficient iff the likelihood splits as $f(\mathbf{x};\theta)=g\!\big(T(\mathbf{x}),\theta\big)\,h(\mathbf{x})$, where $\theta$ enters only through $T$. Every exponential-family law is built this way, $f(x;\theta)=h(x)\exp[\eta(\theta)T(x)-A(\theta)]$, so the sufficient statistic falls straight out of the formula. Conditioning any unbiased estimator on $T$ can only shrink its variance (Rao–Blackwell) — which is exactly why estimators built from $T$ are so efficient. In the sim, the True theta slider sets $\theta$, n sets the sample size, and the cards report $T$, the MLE $=f(T)$, and the Cramér–Rao bound $1/(nI)$.
(1) On the Rao-Blackwell tab, compare the spread of the MLE (which uses all the data through $T$) against using only the first observation — the sufficient-statistic estimator is dramatically tighter. (2) On Estimator Comparison, slide n upward and watch the MLE variance ride right along the Cramér–Rao floor. (3) Switch the Distribution to Poisson or Exponential and confirm the same $T=\sum_i X_i$ keeps working — one summary, many distributions.
Sufficient Statistics — Fisher 1922 / Neyman 1935
$T(X)$ sufficient for $\theta$ iff: $f(\mathbf{x};\theta)=g(T(\mathbf{x}),\theta)\cdot h(\mathbf{x})$ (Factorization Theorem)
$$\boxed{\text{Exponential family: }f(x;\theta)=h(x)\exp\!\left[\eta(\theta)T(x)-A(\theta)\right]}$$
| Distribution | Sufficient Statistic T | MLE from T |
|---|---|---|
| Normal(mu,1) | $\sum x_i$ (= n·xbar) | $\hat\mu=T/n=\bar x$ |
| Bernoulli(p) | $\sum x_i$ (number of 1s) | $\hat p=T/n$ |
| Poisson(lambda) | $\sum x_i$ | $\hat\lambda=T/n=\bar x$ |
| Exponential(lam) | $\sum x_i$ | $\hat\lambda=n/T=1/\bar x$ |
If $\hat\theta$ is unbiased and $T$ is sufficient, then $\tilde\theta=E[\hat\theta\mid T]$ satisfies: (1) $E[\tilde\theta]=\theta$ (still unbiased), (2) $\text{Var}(\tilde\theta)\le\text{Var}(\hat\theta)$. Conditioning on sufficient statistic never increases variance.
Sufficient statistic = no information loss.
T(X) is sufficient if knowing T(X) tells you everything the full data X tells you about theta. Once you have T=sum(Xi), the ordering and individual values of X1,...,Xn are irrelevant for estimating theta. The MLE is always a function of the sufficient statistic in the exponential family.
📖 Casella & Berger — Ch. 6.2