The independence question

You're at a casino. The roulette wheel has landed on red five times in a row.

Our brains are wired to see patterns, even where none exist. Independence is the mathematical way to say "knowing one thing tells you nothing about another."

Defining independence

We've seen independence for events: $A$ and $B$ are independent if $P(A \cap B) = P(A)P(B)$.

For random variables, we extend this idea:

In words: the joint probability factors into the product of individual probabilities.

What independence means

When $X$ and $Y$ are independent:

1. No information transfer: Observing $X$ tells you nothing about $Y$
2. Conditional equals unconditional: $P(Y = y | X = x) = P(Y = y)$
3. The joint PMF factors: $p_{X,Y}(x,y) = p_X(x) \cdot p_Y(y)$

Example: two dice

Roll two fair dice. Let $X$ = first die, $Y$ = second die.

The physical separation of the dice makes them independent. Verifying mathematically:

This factorization works for every pair of values. That's what makes them independent.

Calculation shortcuts

Independence gives us shortcuts that simplify calculations.

Rule 1: expectations multiply

For dependent variables, $E[XY]$ would require knowing the joint distribution. For independent variables, we just multiply the means.

Rule 2: variances add

This is why errors from independent sources add in quadrature. Uncertainties don't compound as badly as you might fear.

For dependent variables, we'd need a covariance term: $\text{Var}(X + Y) = \text{Var}(X) + \text{Var}(Y) + 2\text{Cov}(X, Y)$.

Independence means $\text{Cov}(X, Y) = 0$.

IID: the gold standard

In statistics, we often assume data points are "IID":

Examples:

• Coin flips are IID Bernoulli
• Repeated measurements (done carefully) are IID
• Random samples from a large population are approximately IID

Most statistical procedures assume IID data. When this assumption fails, results can be misleading.

Independence vs. correlation

Key relationships:

• Independent ⟹ Uncorrelated (always true)
• Uncorrelated ⟹ Independent (NOT always true!)

Why independence matters

Simplifies calculations

• Joint distributions factor
• Expectations multiply
• Variances add

Enables statistical inference

• The Law of Large Numbers needs independence
• The Central Limit Theorem needs independence
• Most confidence intervals assume independent samples

Real-world consequences

Assuming independence when it's false leads to underestimating risk. The 2008 financial crisis was partly due to assuming mortgage defaults were independent (they weren't).

Summary

Test your understanding

What's next

You now have the core concepts of random variables: PMFs, CDFs, distributions, and independence. These form the foundation for continuous distributions, the Law of Large Numbers, and the Central Limit Theorem.