From "they go together" to a single number
Taller kids usually weigh a bit more. The more hours you practice, the higher your scores tend to climb. Two things that rise and fall together are correlated. Correlation just turns that gut feeling into one tidy number.
Line up your data in pairs: each person has an $x$ (say, hours studied) and a $y$ (test score). If the big $x$'s tend to come with big $y$'s, the pair "moves together" — a positive correlation. If big $x$'s come with small $y$'s, they move in opposite directions — negative. If there is no pattern at all, they are uncorrelated.
For every point, ask how far $x$ sits above its average and how far $y$ sits above its average, then multiply. Add those products up and you get the co-movement signal $S_{xy}=\sum(x_i-\bar{x})(y_i-\bar{y})$. Points where both values are above (or both below) average add a positive contribution; mismatched points subtract. Divide by each variable's own spread so the score can never blow past 1: $r=S_{xy}/\sqrt{S_{xx}S_{yy}}$, which always lands in $[-1,1]$. A quick number: if study hours and scores have $r=0.8$, then $r^2=0.64$ — about 64% of the variation in scores lines up with study time.
$r$ measures linear association only. A perfect U-shape can give $r=0$ even though $x$ and $y$ are tightly linked — so $r=0$ means "no straight-line trend," not "unrelated." When the relationship bends or outliers lurk, Spearman's $r_s$ ranks the data first and still catches any steady up-or-down trend. And a big $r$ from a tiny sample can be pure luck: the statistic $t=r\sqrt{(n-2)/(1-r^2)}$ checks whether the correlation could be a fluke. In the sim, the ρ slider sets the true correlation the points are drawn from, and n sets how many points you sample.
Set ρ near 0 with $n=5$ and press "New Data" a few times — the measured $r$ swings wildly, because small samples lie. Now set ρ$=0.9$ with $n=300$: the cloud tightens toward a line and $r$ sits near 0.9 on every redraw. Finally open the Anscombe Quartet tab — four datasets share $r\approx0.816$ yet look nothing alike, the classic proof that you must plot first, compute second.
Pearson Correlation — Francis Galton 1888 / Karl Pearson 1896
$$\boxed{r = \frac{\sum_{i=1}^n(x_i-\bar{x})(y_i-\bar{y})}{\sqrt{\sum(x_i-\bar{x})^2\cdot\sum(y_i-\bar{y})^2}} = \frac{S_{xy}}{\sqrt{S_{xx}S_{yy}}}\in[-1,1]}$$
| Symbol | Meaning | Range | Interpretation |
|---|---|---|---|
| $r$ | Sample Pearson correlation | $[-1,1]$ | Strength and direction of LINEAR association |
| $\rho$ | Population correlation | $[-1,1]$ | True correlation; $r$ estimates $\rho$ |
| $r^2$ | Coefficient of determination | $[0,1]$ | Proportion of variance in Y shared with X |
| $S_{xy}$ | Sample covariance (×n-1) | $(-\infty,\infty)$ | $\sum(x_i-\bar{x})(y_i-\bar{y})$ |
| $r_s$ | Spearman rank correlation | $[-1,1]$ | Monotonic (not just linear) association; rank-based |
$r=1$: perfect positive linear relationship. $r=-1$: perfect negative linear. $r=0$: no LINEAR association (but could have nonlinear). $|r|$ measures how tightly points cluster around a line. $r^2$ = proportion of Y's variance "explained" by X.
By the Cauchy-Schwarz inequality: $|S_{xy}|\le\sqrt{S_{xx}\cdot S_{yy}}$, so $r\in[-1,1]$ always. Equality holds iff $y_i=a+bx_i$ exactly (perfect linear relationship).
Under $H_0:\rho=0$: $t=r\sqrt{\frac{n-2}{1-r^2}}\sim t_{n-2}$. Reject at level $\alpha$ if $|t|>t_{n-2,\alpha/2}$.
To test $H_0:\rho=\rho_0$ or construct CIs, use Fisher's $z$-transform: $z=\frac{1}{2}\ln\frac{1+r}{1-r}\approx\mathcal{N}\!\left(\frac{1}{2}\ln\frac{1+\rho}{1-\rho},\frac{1}{n-3}\right)$.
Replace observations with ranks $R_i, S_i$: $r_s=r(\text{ranks})$. Robust to outliers and detects any monotone relationship. Under $H_0$: $t=r_s\sqrt{(n-2)/(1-r_s^2)}\sim t_{n-2}$ approximately.
Data: $(x,y)$: (1,2),(2,4),(3,5),(4,4),(5,5). $\bar{x}=3,\bar{y}=4$.
$S_{xx}=10,\; S_{yy}=6,\; S_{xy}=6$
$$r=\frac{6}{\sqrt{10\cdot6}}=\frac{6}{\sqrt{60}}\approx\mathbf{0.775}$$
Strong positive linear association. $r^2=0.600$: 60.0% of Y's variance shared with X.
t-test: $t=0.775\sqrt{3/(1-0.600)}\approx2.12$, $df=3$. p-value ≈ 0.124 — NOT significant at α=0.05 (with only n=5, even a strong r can fail the test).
"r = 0.9 means X causes Y."
Correlation measures linear association, NOT causation. Ice cream sales and drowning deaths are highly correlated (r≈0.9) — both are caused by hot weather (a confounder). Establishing causation requires: randomized experiments (gold standard), or careful causal inference methods (instrumental variables, regression discontinuity, difference-in-differences). "Correlation is not causation" is perhaps the most important principle in statistical reasoning.
📖 Freedman, Pisani & Purves — Statistics, Ch. 8: "Correlation"
"r = 0 means X and Y are independent."
r=0 means no LINEAR association. A perfect quadratic relationship Y=X² gives r=0 for symmetric X. The Anscombe Quartet tab shows datasets with identical r but wildly different relationships. Independence implies r=0, but r=0 does NOT imply independence (except for bivariate normal distributions).
📖 DeGroot & Schervish — Ch. 4.6
"r = 0.3 is a weak/useless correlation."
Interpreting correlation magnitude depends on context. In psychology, r=0.3 is considered medium. In physics or chemistry, r=0.3 might be unacceptably low. In social science with many confounders, r=0.3 can be practically significant. Always consider: (1) domain norms, (2) effect size measures (r²=0.09 means 9% variance explained), (3) sample size (small n gives unreliable r estimates).
📖 Cohen — Statistical Power Analysis for the Behavioral Sciences
Using Pearson r for non-linear monotonic relationships
✅ Use Spearman rank correlation $r_s$ when the relationship is monotone but nonlinear, or when data contains outliers. Pearson r can severely underestimate association strength for nonlinear monotone relationships. Always plot the data first — scatterplot reveals what type of correlation measure is appropriate.
🔍 Students apply Pearson r to ranked data or clearly curved scatterplots.
Comparing correlation coefficients directly without Fisher z-transformation
✅ r is not normally distributed; its sampling distribution is skewed, especially for |r| near 1. To compare correlations or build CIs, first apply Fisher's z-transform: $z=\tanh^{-1}(r)=0.5\ln((1+r)/(1-r))\sim N$ approximately. Then work with z, transform back at the end.
🔍 Students subtract correlations directly: "r₁=0.8, r₂=0.6, so r₁ is 0.2 units larger" — not valid.
Ignoring outliers' effect on Pearson r
✅ A single outlier can dramatically change r. One outlier in a cloud of uncorrelated points can create r=0.8; one outlier can destroy a true correlation. Always plot the data and check outliers before reporting r. Use Spearman r_s or robust correlation measures when outliers are present.
🔍 Students report r from raw data without checking for influential outlier points.