Discrete Distributions I: Bernoulli and Binomial

The Bernoulli Trial

At the heart of the binomial distribution lies the Bernoulli trial. A Bernoulli trial is a single experiment with exactly two possible outcomes, conventionally labeled "success" and "failure." The probability of success is denoted by \( p \), and the probability of failure is \( q = 1 - p \). Crucially, the trials must be independent, meaning the outcome of one trial does not influence the outcome of any other. For example, flipping a fair coin is a Bernoulli trial with \( p = 0.5 \). In a physics context, consider a detector with an efficiency of 90%. Each time a particle passes through, the detector either triggers (success) or fails to trigger (failure). If we assume the detector's state does not change between particles, each passage is an independent Bernoulli trial with \( p = 0.9 \).

While a single Bernoulli trial is simple, real experiments involve many such trials. If we perform \( N \) independent Bernoulli trials, we are interested in the total number of successes, let us call this random variable \( S \). The variable \( S \) can take any integer value from 0 to \( N \). The probability distribution of \( S \) is the binomial distribution.

The Binomial Probability Mass Function

To find the probability that exactly \( k \) successes occur in \( N \) trials, we must consider two factors: the number of ways to arrange these successes and the probability of any specific arrangement. First, the number of distinct sequences of \( N \) trials containing exactly \( k \) successes is given by the binomial coefficient \( \binom{N}{k} \), which counts the combinations of \( k \) successes chosen from \( N \) positions. Second, for any specific sequence with \( k \) successes and \( N-k \) failures, the probability is \( p^k q^{N-k} \), since the trials are independent.

Combining these, the probability mass function (PMF) for the binomial distribution is:

where \( k = 0, 1, \dots, N \). This formula encapsulates the entire statistical behavior of the experiment. It is important to note that the sum of probabilities over all possible values of \( k \) must equal 1, which is guaranteed by the binomial theorem: \( \sum_{k=0}^{N} \binom{N}{k} p^k (1-p)^{N-k} = (p + (1-p))^N = 1 \).

Mean and Variance

For any random variable, the mean (expectation) and variance are key descriptors of its distribution. For the binomial distribution, these have simple closed-form expressions. The mean number of successes is:

This result is intuitive: if you flip a coin \( N \) times with a 50% chance of heads, you expect \( N/2 \) heads. More generally, the expected value scales linearly with the number of trials.

The variance, which measures the spread of the distribution around the mean, is:

The standard deviation is therefore \( \sigma_S = \sqrt{Np(1-p)} \). Notice that the variance is maximized when \( p = 0.5 \), which corresponds to the most "uncertain" trial (like a fair coin flip). If \( p \) is very close to 0 or 1, the variance is small because the outcome is more predictable.

Connection to the Gaussian Distribution

When \( N \) is large and \( p \) is not too close to 0 or 1, the binomial distribution approximates a Gaussian (normal) distribution. This is a consequence of the Central Limit Theorem. The Gaussian approximation is given by:

This approximation is useful for calculations when \( N \) is large, as it allows us to use continuous calculus techniques. However, for small \( N \) or extreme values of \( p \), the exact binomial formula should be used. The transition from discrete to continuous distributions is a central theme in probability theory and has profound implications for how we model physical systems.