Law of Large Numbers

Compact study note.

Summary

The law of large numbers says sample averages stabilize near the population mean under standard independence and finite-mean assumptions.^[1]

Prerequisites

• IID Random Variables
• Expected Value

Notation and Assumptions

For IID $X_1,X_2,\dots$ with $E[X_i]=\mu$ and finite variance, the sample mean ${\overline{X}}_n=n^{-1}\sum _{i=1}^n{X}_i$ converges to $\mu$ in probability.

Essential Result

For every $\epsilon >0$ , $P(|{\overline{X}}_n-\mu |>\epsilon )\to 0$ like $n\to \mathrm{\infty }$ .

Small Example

The average number of heads per flip in many independent fair coin flips tends toward $0.5$ .

Common Mistakes

• Reading the theorem as guaranteeing short-sample accuracy.
• Thinking convergence removes all randomness at finite $n$ .

Connections

• IID Random Variables
• Central Limit Theorem
• Bernoulli Distribution

References