Picture three coffee shops, and you want to know whether their wait times are really different — or whether you just hit a slow line on the day you visited. Every shop has good days and bad days, so a single visit is noisy. One-way ANOVA ("Analysis of Variance") is a fair way to ask one question: do the groups differ by more than their everyday wobble?
Think of it as a signal-to-noise contest. The signal is how far apart the shops' average waits are. The noise is how much the waits bounce around within each shop. If the gap between the shop averages is large compared with that day-to-day bounce, something real is going on.
We measure the signal with the between-group variation — how far each group mean $\bar{y}_i$ sits from the grand mean $\bar{y}$ — and the noise with the within-group variation — how far each point sits from its own group's mean. Turn each into an average "mean square" and take the ratio: $F=\dfrac{MS_{between}}{MS_{within}}$. An $F$ near $1$ says "these groups look the same"; an $F$ well above $1$ says "the gaps are too big to be luck."
Worked number: suppose three groups average $3.2$, $3.8$, and $4.5$. The spread between them works out to $MS_{between}=2.12$, while the leftover scatter within the groups gives $MS_{within}=0.35$. Then
$$F=\frac{2.12}{0.35}\approx 6.0$$
That is far above $1$, so the group means are probably not all equal.
With $k$ groups and $N$ total points, $MS_{between}=SS_B/(k-1)$ and $MS_{within}=SS_W/(N-k)$, and under the null hypothesis (all true means equal) the ratio follows the $F_{k-1,\,N-k}$ distribution. We reject $H_0$ when the $p$-value falls below $\alpha$. In the sim above, the Effect size slider sets how far apart the true means are (the signal), Within-group σ sets the noise, k and n per group fix the degrees of freedom, and α sets the rejection threshold.
Set Effect size = 0 and watch $F$ hover near $1$ with a large $p$ — pure noise, no real difference. Now slide Effect size up to 4 and watch $F$ shoot up while $p$ collapses. Finally, hold the effect size fixed but crank Within-group σ higher: the same mean gaps now drown in the noise, and $F$ shrinks back toward $1$ — proof that ANOVA judges differences relative to scatter.
One-Way ANOVA — Ronald A. Fisher, 1921
$$\boxed{F=\frac{MS_{between}}{MS_{within}}=\frac{SS_B/(k-1)}{SS_W/(N-k)}\overset{H_0}{\sim}F_{k-1,\,N-k}}$$
| Source | SS | df | MS |
|---|---|---|---|
| Between groups | $\sum_i n_i(\bar{y}_i-\bar{y})^2$ | $k-1$ | $SS_B/(k-1)$ |
| Within groups | $\sum_i\sum_j(y_{ij}-\bar{y}_i)^2$ | $N-k$ | $SS_W/(N-k)$ |
| Total | $\sum_i\sum_j(y_{ij}-\bar{y})^2$ | $N-1$ |
$SS_{total}=SS_{between}+SS_{within}$. Under $H_0$ (all $\mu_i$ equal): $MS_{between}\approx\sigma^2\approx MS_{within}$ so $F\approx1$. When means differ: $MS_{between}$ inflated → $F\gg1$.
$\eta^2=SS_{between}/SS_{total}\in[0,1]$ = proportion of total variance explained by group membership. $\eta^2=0.01$ small, $0.06$ medium, $0.14$ large (Cohen).
Three fertilizer groups, n=5 each. ȳ₁=3.2, ȳ₂=3.8, ȳ₃=4.5. Grand mean=3.833. SS_within=4.2.
$SS_B=5[(3.2-3.833)^2+(3.8-3.833)^2+(4.5-3.833)^2]=5[0.400+0.001+0.445]=4.23$
$F=(4.23/2)/(4.2/12)=2.115/0.35=6.04,\;df=(2,12),\;p\approx0.015\;\to$ Reject H₀
"Significant ANOVA tells us which groups differ."
ANOVA only tells you THAT at least one pair differs — not WHICH pairs. Use Tukey HSD, Bonferroni, or Scheffe post-hoc tests for pairwise comparisons. Running multiple t-tests instead inflates Type I error to ~40% for k=5 groups.
📖 Casella & Berger — Ch. 11.3
"ANOVA tests whether variances are equal — it tests variances."
ANOVA tests equality of MEANS using variance ratios. Large F means group means differ, not variances. Use Levene's test or Bartlett's test to check homoskedasticity (equal variance assumption).
📖 Hogg, McKean & Craig — Ch. 7
Wrong: F = MS_between / MS_total
✅ F = MS_between / MS_within (within-group error, NOT total). MS_total is never used in the F-ratio.
🔍 Confusing MS_within with MS_total.
Wrong df: using (k, N-k) instead of (k-1, N-k)
✅ Numerator df = k-1 (groups minus 1). Denominator df = N-k. Always k-1, not k, for between-groups df.
🔍 Using k instead of k-1 gives wrong critical value.
Not checking homoskedasticity before ANOVA
✅ ANOVA requires equal within-group variances. If violated, use Welch's ANOVA (oneway.test in R) or Kruskal-Wallis nonparametric test. Always plot boxplots to inspect variance spread across groups.
🔍 Applying standard ANOVA to groups with very different spreads.