Axioms of Probability — Series Overview
Combinatorial analysis told us how to count outcomes. To turn counts into probabilities we need a precise, axiomatic foundation. Without one, words like “likely”, “chance”, and “random” mean different things to different people, and proofs become impossible. The axiomatic approach — first nailed down by Kolmogorov in 1933 — gives us three short rules from which every classical probability theorem in this book follows.
This series rebuilds Chapter 2 of Ross's A First Course in Probability. We will define sample spaces and events, state the three axioms, prove the “simple propositions” that every probability problem implicitly uses, build the classical equally-likely-outcomes model, then push the theory into limits and finally into subjective belief.
What This Series Covers
Each post stands alone but builds on the one before it. We move from set-theoretic vocabulary, to the three rules of probability, to the consequences of those rules, to the classical model, to limits, and finally to interpretation.
Sample Spaces and Events
What “outcome” and “event” really mean. Set operations $A \cup B$, $A \cap B$, $A^c$. Mutually exclusive events, De Morgan's laws. Animated Venn diagrams.
PART 3The Three Axioms
$0 \le P(E) \le 1$, $P(S) = 1$, and countable additivity. Why each axiom is necessary, what each one rules out, and why frequentist intuition forces the third.
PART 4Simple Propositions & Inclusion–Exclusion
$P(\emptyset) = 0$, $P(A^c) = 1 - P(A)$, monotonicity, $P(A \cup B) = P(A) + P(B) - P(A \cap B)$, and the general inclusion–exclusion formula.
PART 5Equally Likely Outcomes
Classical probability $P(E) = |E| / |S|$. Cards, dice, balls, the birthday problem — everywhere combinatorial analysis meets the axioms.
PART 6Continuity of Probability
Increasing and decreasing event sequences, $P(\lim E_n) = \lim P(E_n)$, and why countable additivity is exactly the assumption that makes this work.
PART 7Probability as a Measure of Belief
Subjective probability, the Dutch-book argument, and why a rational agent's degrees of belief must obey the same three axioms.
Prerequisites
- Basic set theory: $\in$, $\subseteq$, $\cup$, $\cap$, $A^c$.
- Sigma notation $\sum_{i=1}^{n}$ and limits of sequences.
- Comfort with counting from the Combinatorial Analysis series — we'll use those formulas in Part 5.
Notation Cheat Sheet
| Symbol | Reads as | Means |
|---|---|---|
| $S$ | "sample space" | Set of all possible outcomes of the experiment. |
| $E,F,A,B$ | "events" | Subsets of $S$. We say $E$ “occurs” if the realised outcome lies in $E$. |
| $E^c$ or $\bar E$ | "complement of E" | $S \setminus E$ — the event that $E$ does not occur. |
| $E \cup F$ | "E or F" | Outcomes in $E$, in $F$, or in both. |
| $E \cap F$ or $EF$ | "E and F" | Outcomes in both $E$ and $F$. |
| $E \cap F = \emptyset$ | "mutually exclusive" | $E$ and $F$ cannot both occur. |
| $P(E)$ | "probability of E" | A number in $[0,1]$ assigned to event $E$ obeying the three axioms. |
Why Bother With Axioms?
If you only ever work with finite, equally-likely sample spaces, you can get a long way with the schoolbook formula $P(E) = |E|/|S|$. The trouble starts the moment any of these happens:
- Outcomes are not equally likely (a biased coin, a weighted die).
- The sample space is infinite (waiting times, random numbers in $[0,1]$).
- You need to combine probabilities of complicated events ($P(A \cup B \cup C)$, limits of events).
The axiomatic approach treats $P$ as a function on subsets of $S$ that obeys three rules. Whether outcomes are equally likely, whether $S$ is finite or infinite, whether $P$ comes from physics, data, or belief — everything afterwards is a theorem.
Visual Preview
Here's the kind of step-by-step animation you'll see throughout the series. We start with the sample space of rolling two dice, highlight an event $E = \{\text{sum} = 7\}$, and read off its probability $|E|/|S|$ — a direct application of axiom 2 plus the equally-likely model.
▶ Visual: Probability of “sum equals 7” on two fair dice
Step through the $6 \times 6 = 36$ outcomes of two dice, highlight the diagonal where the sum is $7$, and read off the probability.
How to Use This Series
Read the posts in order if probability is new to you. If you have seen the material before, skim the “Why It Works” sections — that is where the axiomatic intuition lives. Each post ends with worked examples and practice problems drawn from Ross.
Whenever a formula tempts you to memorise it, run the animation, draw the Venn diagram, and convince yourself by hand. Probability rewards careful definitions far more than clever computation.