Probability: Sample Spaces, Axioms and Conditioning

The Logic of Uncertainty

Imagine you are measuring the decay time of a radioactive nucleus. You cannot predict the exact moment it will decay, but you can describe the likelihood of it happening within a certain time window. Probability theory formalises this intuition. It does not tell us what will happen, but it tells us how to assign consistent numerical weights to all possible outcomes. The core idea is that the universe of all possible outcomes is a fixed set, and our measurement is a random selection from that set. The axioms we define next are the non-negotiable rules that ensure these weights behave sensibly, preventing contradictions like the probability of an event being negative or the total probability of all outcomes exceeding one.

Sample Spaces and Events

Every probabilistic experiment begins with the definition of the sample space, denoted by \(\Omega\). This is the set of all mutually exclusive and exhaustive elementary outcomes. For a single measurement of a continuous variable, \(\Omega\) might be the real line \(\mathbb{R}\). For a discrete outcome, such as the result of a detector click, \(\Omega\) might be \(\{0, 1\}\). An event is any subset of \(\Omega\). We assign a probability function \(P\) that maps these events to the interval \([0, 1]\). The function must satisfy three axioms, known as the Kolmogorov axioms, which form the bedrock of modern probability theory.

The third axiom, countable additivity, ensures that if you partition the sample space into infinitely many disjoint events, the sum of their probabilities equals one. This property is crucial for continuous variables, where we integrate over infinitesimal intervals rather than summing discrete points.

Conditional Probability and Independence

In experimental physics, we rarely observe variables in isolation. We often want to know the probability of an outcome given that another condition has been met. This is conditional probability, defined as \(P(A \mid B) = \frac{P(A \cap B)}{P(B)}\), provided \(P(B) > 0\). This concept is the engine behind Bayes' theorem, which allows us to update our beliefs about a parameter based on new data. Two events \(A\) and \(B\) are independent if the occurrence of one does not influence the probability of the other, formally expressed as \(P(A \cap B) = P(A)P(B)\). In measurement contexts, independence often implies that the noise in one channel is uncorrelated with the signal in another, a simplification that is frequently assumed but must be verified experimentally.

From Discrete to Continuous

While discrete probabilities are intuitive, most physical measurements yield continuous values. We cannot assign a non-zero probability to a single exact value in a continuous sample space, as this would violate the additivity axiom for infinite partitions. Instead, we introduce the probability density function (PDF), \(f(x)\). The probability that a variable \(X\) falls within an interval \([a, b]\) is given by the integral \(\int_a^b f(x) \, dx\). The PDF itself is not a probability, but a density; its units are inverse to the units of the variable. This distinction is vital: a high peak in the PDF indicates a high likelihood per unit interval, not a high absolute probability. The total area under the curve must always equal one, preserving the axiom \(P(\Omega) = 1\).