Every morning a bakery's loaves come out at slightly different weights, and suppose you weigh just a handful of them. From those few numbers, what is your single best guess for the typical weight the oven is really aiming for? That is the whole game of estimation: using a small sample of data to pin down a hidden number — a parameter — that you can never observe directly.
The oldest trick is to match averages. Your data has an average you can compute, the sample mean $\bar{x}=\frac{1}{n}\sum_i x_i$, and the model has a "true" average. The method of moments (MOM) just sets them equal and solves for the unknown. Weigh five loaves at $498, 502, 495, 505, 500$ grams: the sample mean is $500$ g, so the MOM estimate of the oven's target is simply $\hat{\mu}=500$ g. One average, one equation, done.
Maximum likelihood asks a sharper question: of all possible parameter values $\theta$, which one makes the data you actually saw the most probable? Each point has a density $f(x_i;\theta)$, and if the points are independent the chance of the whole sample is the product $\prod_i f(x_i;\theta)$. Maximizing that product — in practice by maximizing its logarithm, the log-likelihood $\ell(\theta)=\sum_i\log f(x_i;\theta)$, which turns an awkward product into a friendly sum — gives the estimate
$$\hat{\theta}_{MLE}=\arg\max_\theta\,\ell(\theta)=\arg\max_\theta\sum_i\log f(x_i;\theta).$$
For the normal distribution both recipes land on the same answer, $\bar{x}$; for skewed models like the gamma they part ways and MLE usually wins. On the panel above the sliders set the model's parameters, the dashed curve is the truth your samples are drawn from, and the solid curve is the fitted estimate.
Generate just a few samples and watch the fitted curve wobble far from the dashed true curve. Then add $+1000$ more and see it snap onto the truth — that is consistency, the promise that the estimate homes in on the right value as data piles up. Finally, watch how tightly the fitted peak tracks the true peak: MLE squeezes the most information out of every observation, which is why it is the gold-standard estimator for large samples.
Point Estimation — MLE & MOM — Maximum Likelihood and Method of Moments
$$\hat\theta_{MLE} = \arg\max_\theta \ell(\theta;\mathbf{x}) = \arg\max_\theta \sum_i\log f(x_i;\theta)$$
$\text{MOM: set } E[X^k]=\frac{1}{n}\sum x_i^k, \text{ solve for } \theta$
This distribution has important theoretical and applied properties. See the simulation tabs to explore how shape changes with parameters.
Parameter estimation for any statistical model. MLE: logistic regression, linear regression (= OLS under normality), mixture models, survival models. MOM: quick estimates, method of moments for generalized method of moments (GMM) in econometrics. MLE has optimal large-sample properties (efficiency); MOM is computationally simpler but less efficient.
Use the interactive simulation above to explore how the PDF shape responds to parameter changes. The Live Sampling tab demonstrates convergence of empirical frequencies to theoretical probabilities.
Always verify parameter definitions before computing probabilities.
Different textbooks and software packages use different parameterizations. Always check which convention is being used (e.g., rate vs scale for exponential/gamma; shape vs rate for beta). Plug in extreme values and check that the PDF shape matches your expectation.
See the simulation to visualize how parameters affect the distribution shape.