Total Probability and Bayes' Theorem

Compact study note.

Summary

The law of total probability decomposes events across partitions. Bayes' theorem reverses conditional probability by combining likelihood with prior probabilities.^[1]

Prerequisites

Conditional Probability

Notation and Assumptions

Let $B_{1}, \dots, B_{n}$ be disjoint events with positive probability and union $Ω$ . Then

P (A) = \sum_{i} P (A ∣ B_{i}) P (B_{i}) .

Essential Result

Bayes' theorem, whenever $P (A) > 0$ :

P (B_{j} ∣ A) = \frac{P (A ∣ B_{j}) P (B_{j})}{\sum_{i} P (A ∣ B_{i}) P (B_{i})} .

Small Example

If a test is positive with probability $0.9$ for disease and $0.05$ without disease, and prevalence is $0.01$ , then $P (D ∣ +) = 0.009 / (0.009 + 0.0495) \approx 0.154$ .

Common Mistakes

Ignoring base rates.
Using Bayes' theorem when conditioning events do not form partition.

Connections

References

OpenStax, Introductory Statistics 2e, "Chapter 3: Probability Topics", https://openstax.org/books/introductory-statistics-2e/pages/3-introduction ↩︎