From "what's the weather tomorrow?" to stationary distributions
Picture the weather as a game with a short memory. To guess tomorrow, you only need to know today — sunny or rainy. It doesn't matter that it poured all last week. "Only the present matters for the future" is the entire idea of a Markov chain, and that one rule (called the Markov property) is what makes these chains so easy to work with.
Each situation is a state — here just State 1 and State 2. The chance of hopping from one state to another is a fixed number, the transition probability $p_{ij}$ (chance of going from state $i$ to state $j$). If the chance of staying in State 1 is $p_{11}=0.7$, then the chance of leaving for State 2 is $1-0.7=0.3$. Stack all of these into a transition matrix $\mathbf{P}$ where every row adds to $1$ — because from any state you must land somewhere next step.
A worked number: start in State 1 with $p_{11}=0.7,\ p_{22}=0.6$. After one step the probability of being in State 1 is just $0.7$. After two steps it is $0.7\times0.7+0.3\times(1-0.6)=0.49+0.12=0.61$. Notice it is already drifting away from where it started — and it keeps drifting toward one settled value.
That settled value is the stationary distribution $\boldsymbol{\pi}$: the one row vector that doesn't change when you multiply by the matrix, $\boldsymbol{\pi}\mathbf{P}=\boldsymbol{\pi}$ with $\sum_i\pi_i=1$. For two states there is a tidy closed form, $\pi_1=\dfrac{1-p_{22}}{(1-p_{11})+(1-p_{22})}$. With our numbers $\pi_1=\dfrac{0.4}{0.3+0.4}=\dfrac{0.4}{0.7}\approx0.571$. How fast the chain reaches $\boldsymbol{\pi}$ is set by the second eigenvalue $\lambda_2=p_{11}+p_{22}-1$: the closer $|\lambda_2|$ is to $1$, the slower it mixes. The two sliders set $p_{11}$ and $p_{22}$, and those two numbers alone fix the whole matrix, the stationary $\boldsymbol{\pi}$, and the mixing speed.
Push both sliders high (near $0.95$) and open the Mixing Time tab — the decay curve crawls, because $|\lambda_2|$ sits near $1$. Make the chain symmetric ($p_{11}=p_{22}$) and check the Convergence tab: both starting states settle at a 50/50 split. Finally, set Steps near $2000$, run the chain, and compare the "Emp. freq(1)" card to $\pi_1$ — watching the long-run frequency match the stationary probability is the ergodic theorem in action.
Markov Chains — Andrei Markov, 1906
Markov property: $P(X_{t+1}=j\mid X_t=i,X_{t-1},\ldots)=P(X_{t+1}=j\mid X_t=i)=p_{ij}$
$$\boxed{\text{Stationary distribution: }\boldsymbol{\pi}\mathbf{P}=\boldsymbol{\pi},\quad\sum_i\pi_i=1}$$
2-state: $\pi_1=\dfrac{1-p_{22}}{(1-p_{11})+(1-p_{22})}$
For irreducible aperiodic chains: time averages converge to ensemble averages: $\frac{1}{T}\sum_{t=1}^T f(X_t)\to E_\pi[f]$ a.s. This justifies MCMC sampling.
Convergence rate governed by second eigenvalue $\lambda_2=p_{11}+p_{22}-1$. TV distance $\le |\lambda_2|^t/2$. Mixing time $\approx 1/(1-|\lambda_2|)=$ spectral gap inverse.
Assuming Markov chains always converge to stationary distribution.
Convergence requires: (1) Irreducibility — every state reachable from every other. (2) Aperiodicity — gcd of return times = 1. Absorbing states violate (1). Two-cycle chains violate (2). Also: not checking if empirical frequencies match stationary distribution after short runs — need sufficient mixing time first.