The Binomial Distribution from Bernoulli Trials

Derivation

The Binomial Distribution from Bernoulli Trials

D-115 Home PU-106 Threads chance Depends on Expectation, Variance and Their Algebra

Statement

Let S = X₁ + … + X_N be the number of successes in N independent, identically distributed Bernoulli trials, each success carrying probability p ∈ [0,1] and failure probability q = 1 − p. Then S is binomially distributed with probability mass function P(S = k) = ^NC_k p^k q^N−k for k = 0, 1, …, N, with mean Np and variance Np(1 − p).

Why it matters

The binomial distribution is the exact law of a count of successes whenever a fixed number of independent yes/no trials share a common success probability — radioactive counts in a fixed window with small dead time, photon detections at a photodiode, spin-up measurements on identically prepared qubits, or lattice-site occupations. It is the discrete workhorse from which the Poisson and Gaussian laws both descend as limits.

Because it is built entirely from the sum of independent Bernoulli variables, it is also the cleanest laboratory for the two structural theorems of probability: linearity of expectation (which needs no independence) and additivity of variance (which does). Seeing exactly where each theorem enters teaches you what independence buys you.

Assumptions

Fixed trial count N.If N is itself random (e.g. the observation window admits a random number of events), the count is a compound distribution, not binomial — a Poisson number of Bernoulli trials gives a Poisson success count, not a binomial one. Two outcomes per trial.If each trial has three or more outcomes, the joint counts follow the multinomial distribution; the single-count marginal is still binomial, but joint statistics are not captured. Common success probability p.If p varies trial-to-trial (a Poisson–binomial situation), the mean is still Σp_i but the variance becomes Σp_i(1−p_i) and the pmf loses its closed binomial-coefficient form.

Mutual independence of the trials.Linearity of expectation survives dependence, so the mean stays Np; but the variance acquires cross-covariance terms 2Σ_i<jCov(X_i,X_j), so positively correlated trials (clumping) inflate the variance above Np(1−p) and the pmf is no longer binomial.

Derivation

Part I — the probability mass function.

X_i ∈ {0,1}, P(X_i=1) = p, P(X_i=0) = q = 1−p

Definition of a Bernoulli trial: a single indicator variable. Encoding success as 1 and failure as 0 makes the count the sum S = ΣX_i. A

P(a specific ordered sequence with k ones) = p^k q^N−k

By independence the joint probability factorises into the product of the N single-trial probabilities; k factors of p and N−k factors of q. The value does not depend on the order of the ones. A

# distinct sequences with exactly k ones = ^NC_k = N! / [k!(N−k)!]

Counting the ways to choose which k of the N positions are successes — a combination, since the successes are indistinguishable. A

P(S=k) = Σ_sequences p^kq^N−k = ^NC_k p^k q^N−k

The event {S=k} is the disjoint union of all sequences with k ones; probabilities of mutually exclusive events add, and every term is equal, so multiply the common term by the count from step 3. A

Σ_k=0^N ^NC_k p^kq^N−k = (p+q)^N = 1^N = 1

Normalisation check by the binomial theorem — this is why the distribution is named “binomial.” Since p+q=1 the total probability is 1, confirming a valid pmf. B

Part II — the mean.

E[X_i] = 1·p + 0·q = p

Expectation of a single indicator from its definition Σ (value × probability). The prior result on moments gives E[X]=Σ_xx P(x). A

E[S] = E[Σ_i=1^NX_i] = Σ_i=1^N E[X_i] = Np

Linearity of expectation, which holds for any random variables whether or not they are independent; each of the N identical terms equals p. B

Part III — the variance.

E[X_i²] = 1²·p + 0²·q = p ⇒ Var(X_i) = E[X_i²] − E[X_i]² = p − p² = p(1−p)

Because X_i only takes values 0 and 1, X_i² = X_i, so E[X_i²]=p; then apply Var = E[X²]−(E[X])² from the prior result. B

Var(S) = Var(Σ_iX_i) = Σ_iVar(X_i) + 2Σ_i<jCov(X_i,X_j)

General expansion of the variance of a sum. This is the exact identity before using independence, exposing precisely where independence will be spent. C

independence ⇒ Cov(X_i,X_j) = 0 for i≠j ⇒ Var(S) = Σ_i=1^N p(1−p) = Np(1−p)

Pairwise independence makes every cross-covariance vanish, so only the N equal diagonal terms survive. This is the one step that genuinely uses independence. C

Result

P(S=k) = ^NC_k p^k(1−p)^N−k, E[S] = Np, Var(S) = Np(1−p)

Reading. The chance of exactly k successes is the probability of one favourable arrangement, p^kq^N−k, multiplied by the number of favourable arrangements, ^NC_k. The count drifts around Np; its spread, measured by the standard deviation √[Np(1−p)], is largest at p=½ and pinches to zero as p approaches 0 or 1.

Units check. Probabilities and counts are dimensionless: p, q are pure numbers, ^NC_k is a pure integer, so P is dimensionless as required. The mean Np and standard deviation √[Np(1−p)] both carry units of “counts,” the same units as S itself — consistent.

Limiting cases

N=1: P(0)=q, P(1)=p; mean p, variance pq — recovers the single Bernoulli trial.
p→0 or p→1: the distribution collapses to a point mass at k=0 or k=N; variance Np(1−p)→0, no fluctuation.
p=½: symmetric about N/2, variance maximal at N/4 — the noisiest, most uncertain case.
N→∞, Np=λ fixed: Poisson limit, P(k)→e^−λλ^k/k! with mean = variance = λ.
N→∞, p fixed: de Moivre–Laplace / central-limit Gaussian, S ≈ 𝒩(Np, Np(1−p)).

Breaks when

Trials are correlated. Sampling without replacement from a finite urn makes successive draws dependent; the correct law is hypergeometric, and the variance is reduced by the finite-population factor (M−N)/(M−1) below Np(1−p). Positive correlation (clumping/contagion) instead inflates the variance (overdispersion).
The success probability varies across trials. With per-trial p_i the count is Poisson–binomial: the closed ^NC_k form is lost, and although the mean is still Σp_i, the variance Σp_i(1−p_i) is always ≤ the equal-p value.
N is random. If the number of trials is itself a random variable (e.g. events arriving in a fixed time window), the count is a compound distribution; a Poisson-distributed N yields a Poisson count with variance = mean, breaking the Np(1−p) < Np relation.
More than two outcomes per trial. Categorical trials give the multinomial law; the binomial applies only to a single lumped “success vs. everything else” margin.

Failure modes

Dropping the binomial coefficient. Writing P(S=k) = p^kq^N−k gives the probability of one specific ordered sequence, not of the event “k successes in any order.” The multiplicities ^NC_k are exactly what fail to sum to 1.
Using N permutations instead of ^NC_k combinations. Successes are indistinguishable; dividing out k! is essential. Confusing ^NP_k with ^NC_k overcounts by k!.
Adding standard deviations instead of variances. Only variances of independent variables add; SD(S)=√[Np(1−p)], not N√[p(1−p)]. The standard deviation grows like √N, so the relative spread SD/mean shrinks like 1/√N.
Assuming variance = mean. That equality is the Poisson property; for a binomial Var = Np(1−p) < Np whenever p>0 (underdispersion relative to Poisson).
Forgetting independence in the variance. Reusing Var(S)=Np(1−p) for correlated trials silently discards the covariance terms and can be badly wrong.
Off-by-one on the range. k runs 0 to N inclusive — that is N+1 values, not N.

Discussion

The decomposition S=ΣX_i is the whole story. The mean uses only linearity of expectation and therefore never needed the trials to be independent — the average number of heads in N tosses is Np even for a machine that deliberately correlates its tosses. The variance is where independence is spent: the cross-covariance terms in step 9 vanish only because the trials do not talk to each other. This asymmetry — mean robust, variance fragile — recurs throughout statistical physics, where correlations leave the average energy unchanged but reshape the fluctuations.

The variance Np(1−p) drives the √N scaling of statistical noise. Since SD ∝ √N while the mean ∝ N, the fractional uncertainty SD/mean = √[(1−p)/(Np)] falls as 1/√N. This is the quantitative root of the law of large numbers and of why counting experiments improve only as the square root of the sample size — to halve the relative error you must quadruple the counts.

The binomial sits at the head of a family generated by two orthogonal limits. Holding p fixed and sending N→∞, the standardised variable (S−Np)/√[Np(1−p)] converges to the standard normal (de Moivre–Laplace, the earliest central limit theorem). Holding the product Np=λ fixed while p→0 instead sends it to the Poisson law, where the variance-to-mean ratio locks to 1. That the same distribution flows to a symmetric continuous bell in one limit and a skewed discrete rare-event law in another is a compact map of how macroscopic regularity emerges from microscopic chance. The natural exponential-family / maximum-entropy characterisation (binomial as the max-entropy law for a fixed mean count over a bounded support) makes these limits structural rather than coincidental.

Common misconceptions. The binomial does not require p=½ — that is only the symmetric special case. “Independent” refers to the trials, not the successes-vs-failures within a trial. And a run of successes does not lower the probability of the next success: independence means the trials carry no memory (the gambler’s fallacy). Finally, the mode (most likely k) is ⌊(N+1)p⌋, which is near but not identical to the mean Np.

Worked examples

Example 1 — qubit readout. A qubit is prepared and measured N=10 times, each measurement independently returning “1” with probability p=0.3. Find P(exactly 4 ones), the mean, and the standard deviation of the total count.

P(S=k) = ^NC_k p^k(1−p)^N−k, k=4

Apply the derived pmf; symbols first. A

¹⁰C₄ = 210, p⁴=(0.3)⁴=0.0081, (0.7)⁶=0.117649

Evaluate the combinatorial and power factors numerically. A

P(S=4) = 210 × 0.0081 × 0.117649 = 0.2001

Multiply the three factors. A

E[S]=Np=10×0.3=3.0, Var=Np(1−p)=10×0.3×0.7=2.1, SD=√2.1=1.449

Insert into the derived moment formulae. A

P(S=4) ≈ 0.200, E[S]=3.0 counts, SD ≈ 1.45 counts

Reading. Four ones out of ten is the single most likely outcome (mode ⌊11×0.3⌋=3, and k=4 is close behind), occurring about one run in five; typical counts land within roughly 3 ± 1.5.

Example 2 — radioactive counting. In a fixed gate a detector registers each of N=200 incident particles independently with efficiency p=0.05. Give the expected count, its standard deviation, the fractional uncertainty, and the probability of zero counts.

E[S]=Np, SD=√[Np(1−p)], SD/E = √[(1−p)/(Np)]

Write the symbolic mean, spread, and relative spread from the result. B

E[S]=200×0.05=10.0, Var=200×0.05×0.95=9.5, SD=√9.5=3.082

Substitute the numbers. A

SD/E = 3.082/10.0 = 0.308 (≈ 31%)

Divide; note it is close to 1/√10=0.316, the Poisson estimate, because p is small. B

P(S=0) = (1−p)^N = (0.95)²⁰⁰ = e^{200 ln 0.95} = e^−10.26 = 3.5×10⁻⁵

The k=0 term of the pmf; compare with the Poisson approximation e^−λ=e⁻¹⁰=4.5×10⁻⁵. B

E[S]=10 counts, SD ≈ 3.08 counts, SD/E ≈ 31%, P(0) ≈ 3.5×10⁻⁵

Reading. With efficiency well below 1 the binomial closely tracks a Poisson of mean 10, whose hallmark √N-type ±√10 ≈ ±3.2 counting error the exact binomial slightly undercuts because the (1−p) factor shaves the variance from 10 to 9.5.

Units check. All quantities are counts or dimensionless ratios; SD/E is a pure number, correctly reported as a percentage.

Problems

A fair coin (p=½) is tossed N=6 times. Find P(exactly 3 heads) and the variance of the head count.
Solution
P(3)=⁶C₃(½)³(½)³=20×(1/8)×(1/8)=20/64=0.3125. Variance=Np(1−p)=6×½×½=1.5 counts² (SD=1.225).
A biased die shows a six with probability p=1/6 on each of N=12 rolls. Compute the expected number of sixes and the probability of no sixes.
Solution
E[S]=Np=12×(1/6)=2.0 sixes. P(0)=(5/6)¹²=e^{12 ln(0.8333)}=e^−2.188=0.112, i.e. about an 11% chance of zero sixes.
Prove directly from the pmf, using k ^NC_k = N ^N−1C_k−1, that E[S]=Np, without invoking linearity of expectation.
Solution
E[S]=Σ_k=0^N k ^NC_kp^kq^N−k. The k=0 term vanishes; for k≥1 substitute k ^NC_k=N ^N−1C_k−1 and factor one p: E[S]=Np Σ_k=1^N ^N−1C_k−1p^k−1q^N−k. Let j=k−1: the sum is Σ_j=0^N−1 ^N−1C_jp^jq^N−1−j=(p+q)^N−1=1. Hence E[S]=Np.
A quality line produces items with defect probability p=0.02, sampled independently in batches of N=50. Find the probability a batch contains at least one defect, and the mean and SD of the defect count.
Solution
P(≥1)=1−P(0)=1−(0.98)⁵⁰=1−e^50 ln0.98=1−e^−1.010=1−0.364=0.636. Mean=Np=50×0.02=1.0 defect; Var=Np(1−p)=50×0.02×0.98=0.98, SD=√0.98=0.990.
For fixed N, show that the variance Np(1−p) is maximised at p=½ and state the maximum. Then, for N=100, compare the SD at p=0.5 with that at p=0.1.
Solution
Differentiate f(p)=Np(1−p)=N(p−p²): f′=N(1−2p)=0 ⇒ p=½; f″=−2N<0 confirms a maximum, value N/4. At N=100: p=0.5 gives Var=25, SD=5.0; p=0.1 gives Var=100×0.1×0.9=9, SD=3.0. The fair case is the noisiest, with 5/3≈1.67× the standard deviation.