The equilibrium

Run a Markov chain long enough and the distribution over states stops changing. No matter where you started.

What do you think?

A Markov chain has states A, B, C. You start at A and run it for 10,000 steps. Your friend starts at C and runs it for 10,000 steps. Will you visit the states in the same proportions?

Stationary distribution

Stationary Distribution

A probability vector $\boldsymbol\pi$ is a stationary distribution of a Markov chain with transition matrix $P$ if: $\boldsymbol\pi = \boldsymbol\pi P$ and $\sum_i \pi_i = 1$ . Once the chain reaches this distribution, it stays there forever.

Think of it as a balance condition: the probability flowing into each state equals the probability flowing out.

Finding the stationary distribution

Weather chain stationary distribution

\begin{cases} \pi_S = 0.8\pi_S + 0.4\pi_R \\ \pi_R = 0.2\pi_S + 0.6\pi_R \end{cases}

Write the balance equations π = πP

Step 1 of 4

In the long run, it's sunny 2/3 of the time and rainy 1/3 — regardless of today's weather.

Watch it converge

Pour "probability fluid" into the nodes. Watch it flow through the transitions until the levels stabilize at the stationary distribution:

Stationary Distribution Flow

Start from

Step

π(Sun) target

0.6667

π(Rain) target

0.3333

|Error|

0.333333

The convergence theorem

Convergence Theorem

If a Markov chain is irreducible and aperiodic, then:

A unique stationary distribution $\boldsymbol\pi$ exists.
For any starting distribution, $P(X_n = j) \to \pi_j$ as $n \to \infty$ .
The fraction of time spent in state $j$ converges to $\pi_j$ .

Reducible chains may have multiple stationary distributions. Periodic chains cycle rather than converge. Both conditions (irreducible + aperiodic) are needed.

Detailed balance

A stronger condition than stationarity:

Detailed Balance

A distribution $\boldsymbol\pi$ satisfies detailed balance (is "reversible") if: $\pi_i \, p_{ij} = \pi_j \, p_{ji} \quad \text{for all } i, j$ The flow from $i$ to $j$ equals the flow from $j$ to $i$ .

Detailed balance implies stationarity (but not vice versa). MCMC algorithms are built on it.

Applications

Domain	States	Transitions
Page ranking	Web pages	Links between pages
Genetics	Alleles	Mutation + selection
Speech recognition	Phonemes	Language model
Finance	Market regimes	Bull ↔ Bear transitions
Board games	Board positions	Dice rolls + moves

Practice problems

The stationary distribution satisfies π = πP. What does this mean in words? (one word)

For the weather chain (π_S = 2/3, π_R = 1/3), what fraction of a 300-day span is rainy? (whole number)

Summary

Concept	Key Formula
Stationary distribution	$\boldsymbol\pi = \boldsymbol\pi P$ , $\sum \pi_i = 1$
Convergence	Irreducible + aperiodic → unique $\boldsymbol\pi$
Time averages	Fraction in state $j$ → $\pi_j$
Detailed balance	$\pi_i p_{ij} = \pi_j p_{ji}$

The stationary distribution is the Markov chain's version of the LLN. Just as sample averages converge to the true mean, state visit frequencies converge to $\boldsymbol\pi$ .

What's next

The stationary distribution is a theoretical target. But what if we can't solve $\boldsymbol\pi = \boldsymbol\pi P$ analytically? MCMC algorithms construct a Markov chain whose stationary distribution is exactly the distribution we want to sample from.