Bessel's Correction for Sample Variance

Derivation

Bessel's Correction for Sample Variance

D-113 Home PU-106 Threads chance Depends on Expectation, Variance and Their Algebra

Statement

Let X₁, …, X_N be independent, identically distributed random variables with finite mean μ = E[X] and finite variance σ² = Var(X). Define the sample mean X̄ = (1/N) ∑_i X_i and the sample variance s² = (1/(N−1)) ∑_i (X_i − X̄)². Then E[s²] = σ² exactly, for every N ≥ 2. The divisor N−1 — Bessel's correction — is precisely what makes the estimator unbiased once the mean has been estimated from the same data.

Why it matters

Almost every experimental error bar in physics is a sample standard deviation computed from a finite run of measurements whose true mean is unknown. If we naively divided the sum of squared residuals by N, we would systematically underestimate the spread of the parent distribution — and therefore quote error bars that are too small, on average, by a factor (N−1)/N. For a handful of repeat measurements this bias is large: with N = 4 it is a full 25%.

The correction is also the cleanest concrete instance of a deep statistical principle: fitting a parameter to data consumes one degree of freedom. The residuals about a fitted mean are not free to roam — they are constrained to sum to zero — and the divisor must be reduced to account for exactly that lost freedom. The same bookkeeping reappears in least-squares fitting and in the χ² distribution.

Assumptions

Finite second moment, E[X²] < ∞.If σ² is undefined or infinite (e.g. a Cauchy parent), E[s²] does not exist and there is nothing to be unbiased about.

Independent, identically distributed draws.Correlated samples make E[(X̄−μ)²] ≠ σ²/N; the cross-terms no longer vanish and the N−1 divisor stops removing the bias exactly.

The mean μ is unknown and estimated by X̄.If the true μ were known and used in place of X̄, then (1/N)∑(X_i−μ)² is already unbiased and one divides by N, not N−1. The correction exists solely because the centring point is itself fitted.

No requirement of normality.Unbiasedness of s² holds for any distribution with finite variance. Normality would only be needed for stronger claims (e.g. the sampling distribution of s² being scaled χ²), not for this result.

Derivation

∑_i=1^N (X_i − X̄)² = ∑_i [(X_i − μ) − (X̄ − μ)]²

Insert and subtract the true mean μ; a pure algebraic identity, valid termwise. A

= ∑_i (X_i−μ)² − 2(X̄−μ) ∑_i (X_i−μ) + N(X̄−μ)²

Expand the square and pull the constant (X̄−μ) out of the sum over i. A

∑_i (X_i−μ) = N(X̄−μ)

Direct from the definition X̄ = (1/N)∑_i X_i, i.e. ∑_i X_i = N X̄. A

∑_i (X_i−X̄)² = ∑_i (X_i−μ)² − N(X̄−μ)²

Substitute step 3 into the middle term: −2(X̄−μ)·N(X̄−μ) + N(X̄−μ)² = −N(X̄−μ)². This is the key decomposition. B

E[∑_i (X_i−X̄)²] = ∑_i E[(X_i−μ)²] − N E[(X̄−μ)²]

Take expectations of both sides; linearity of E[·] distributes over the sum and the difference. B

E[(X_i−μ)²] = σ² ⇒ ∑_i E[(X_i−μ)²] = Nσ²

By definition of population variance for each identically distributed X_i (prior result: moments & variance). A

E[(X̄−μ)²] = Var(X̄) = Var ( (1/N)∑_i X_i ) = (1/N²) ∑_i Var(X_i) = σ²/N

Since E[X̄] = μ, the mean-squared deviation of X̄ is its variance. Independence kills all cross-covariances, so the variance of the sum is the sum of variances. C

E[∑_i (X_i−X̄)²] = Nσ² − N·(σ²/N) = (N−1)σ²

Substitute steps 6 and 7 into step 5. The estimated centre carries away exactly one unit of σ². B

E[s²] = E [ (1/(N−1)) ∑_i (X_i−X̄)² ] = (N−1)σ² / (N−1) = σ²

Divide the previous line by the constant N−1. The correction factor and the residual expectation cancel identically. A

Result

E[s²] = σ², where s² = (1/(N−1)) ∑_i=1^N (X_i − X̄)²

Reading. Averaged over all possible samples of size N, the Bessel-corrected sample variance equals the true population variance exactly — no bias, for any parent distribution with finite variance. Dividing instead by N would give E = σ²(N−1)/N < σ², an underestimate. The single missing unit in N−1 is the degree of freedom spent locating X̄.

Units check. Each (X_i−X̄)² has units of [X]²; summing N of them and dividing by the dimensionless count N−1 preserves [X]², matching σ² = [X]². Both sides carry the squared units of the measured quantity.

Limiting cases

Known mean. If μ is used instead of X̄, step 4's correction term vanishes and (1/N)∑(X_i−μ)² is unbiased — divisor N, not N−1.
Large N. The bias factor (N−1)/N → 1; the two divisors converge and the distinction becomes numerically negligible.
N = 2. The smallest admissible case: s² = (X₁−X₂)²/2, and indeed E[s²] = σ².
N = 1. Divisor is zero: s² is undefined (0/0). A single point carries no information about spread — correctly reflected.

Breaks when

Heavy-tailed parents with infinite variance (e.g. Cauchy, or a power law with tail index ≤ 2): σ² does not exist, so E[s²] is undefined and Bessel's correction cannot rescue an unbiased estimate.
Correlated or autocorrelated data (time series, spatially adjacent detector channels): Var(X̄) ≠ σ²/N because covariance cross-terms survive step 7. The correct divisor becomes an effective sample size N_eff < N, generally non-integer.
Estimating the standard deviation itself. Unbiasedness of s² does not transfer to s = √s²: by Jensen's inequality E[s] < σ, so s is a biased (slightly low) estimator of σ even with the N−1 divisor.
Multiple fitted parameters. If the centring is a fit with p free parameters rather than a single mean, the residuals lose p degrees of freedom and the unbiased divisor is N−p, not N−1.

Failure modes

"Divide by N−1 always." Applying the correction when the mean is known a priori over-corrects and makes the estimator biased high — the correct divisor is then N.
Correcting s the same way as s². Assuming s = √s² is unbiased for σ. It is not; the square root introduces a downward bias of order 1/N.
Mistaking "unbiased" for "smaller error". The N-divisor estimator actually has smaller mean-squared error in some cases; unbiasedness and minimum variance are different optimality criteria.
Using N−1 on correlated samples. Treating autocorrelated measurements as independent and still dividing by N−1, badly underestimating σ² of the mean.
Confusing σ² (spread of one measurement) with σ²/N (spread of the mean) when quoting the uncertainty on X̄.

Discussion

The physical content of the derivation lives entirely in step 4: the sum of squared deviations about the sample mean is always smaller than the sum about the true mean, by exactly N(X̄−μ)². This is not an accident of algebra — the sample mean is the value of c that minimises ∑(X_i−c)². Because X̄ is fitted to hug the data, residuals measured from it are systematically too small, and the correction inflates them back to their honest expected size.

The language of degrees of freedom makes this precise. The N residuals X_i−X̄ satisfy one linear constraint, ∑_i(X_i−X̄) = 0, so only N−1 of them are independent. The divisor counts independent residuals, not raw data points. Fit p parameters and you impose p constraints, giving the general unbiased divisor N−p — the same N−p that appears as the degrees of freedom of the χ² distribution in a least-squares goodness-of-fit test.

A sharper way to see the "one lost dimension": write the data as a vector in ℝ^N. Subtracting X̄ projects it orthogonally onto the (N−1)-dimensional subspace perpendicular to the all-ones vector (1,…,1). Under a normal parent the projected residual vector is isotropic Gaussian in that subspace, so ∑(X_i−X̄)²/σ² ∼ χ²_N−1, whose mean is N−1 — recovering step 8 geometrically and exposing why the correction is exactly one, independent of the parent's shape (only unbiasedness, not the χ² law, survives non-normality).

Common misconceptions. Bessel's correction is not a small-sample "fudge factor" that becomes wrong for large N — it is exactly unbiased for every N ≥ 2. It also does not require the data to be normally distributed; normality is a separate assumption needed only for the sampling distribution of s², never for its mean.

Worked examples

Data: {2, 4, 4, 4, 5, 5, 7, 9} cm (N = 8)

Eight length measurements; compute the unbiased sample variance and compare against the naive N-divisor value. A

X̄ = (2+4+4+4+5+5+7+9)/8 = 40/8 = 5 cm

Symbolic definition first, then arithmetic. A

∑(X_i−X̄)² = 9+1+1+1+0+0+4+16 = 32 cm²

Squared residuals about the sample mean. A

s² = 32/(8−1) = 32/7 ≈ 4.571 cm² vs 32/8 = 4.000 cm²

Bessel divisor N−1 = 7 against the naive N = 8. A

s² = 32/7 ≈ 4.57 cm², s ≈ 2.14 cm

Reading. The naive estimate 4.00 cm² is 12.5% too low — exactly the factor (N−1)/N = 7/8. Only s² = 4.57 cm² is unbiased for σ².

Units check. Residuals in cm, squared to cm², divided by a pure number → cm²; the standard deviation returns to cm.

Verify unbiasedness numerically for a fair coin: X ∈ {0,1}, p = ½, N = 2.

A Bernoulli parent with known σ² = p(1−p) = ¼ lets us check E[s²] = σ² by exhaustive enumeration. B

s² = (X₁−X₂)²/(2−1) = (X₁−X₂)²

For N = 2 the corrected variance reduces to the squared difference. A

Outcomes (00): s²=0 (01): 1 (10): 1 (11): 0, each with prob ¼

The four equally likely samples of size 2. A

E[s²] = ¼(0) + ¼(1) + ¼(1) + ¼(0) = ½ · 1 = ¼

Average over the sampling distribution. B

E[s²] = ¼ = p(1−p) = σ² ✓

Reading. Exhaustive enumeration confirms the theorem exactly: the N−1 divisor recovers σ² = ¼. Had we divided by N = 2 we would have obtained E = 1/8, half the true variance.

Units check. The coin variable is dimensionless, so s² and σ² are pure numbers — consistent.

Problems

(A) Five voltage readings give {10, 12, 11, 13, 14} V. Compute X̄, the Bessel-corrected s², and s.
Solution
X̄ = 60/5 = 12 V. Residuals: −2, 0, −1, 1, 2 → squares 4, 0, 1, 1, 4, sum = 10 V². s² = 10/(5−1) = 2.5 V², so s = √2.5 ≈ 1.58 V.
(A) For the data in Problem 1, what would the naive N-divisor estimate be, and by what percentage does it underestimate s²?
Solution
Naive value = 10/5 = 2.0 V². Ratio to corrected value = 2.0/2.5 = 0.8 = (N−1)/N = 4/5, i.e. 20% too low.
(B) Show algebraically that ∑_i(X_i−X̄)² = ∑_iX_i² − N X̄², the computational form of the numerator.
Solution
Expand: ∑(X_i−X̄)² = ∑X_i² − 2X̄∑X_i + N X̄². Since ∑X_i = N X̄, the middle term is −2X̄(N X̄) = −2N X̄². Adding +N X̄² gives ∑X_i² − N X̄². ✓
(B) A student measures the parent mean μ = 20.0 s independently and known, then takes N = 6 timings {19, 21, 20, 22, 18, 20} s. Which divisor gives an unbiased variance estimate here, and what is its value?
Solution
Because μ is known, centre on μ = 20, not on X̄, and divide by N = 6. Residuals from 20: −1, 1, 0, 2, −2, 0 → squares 1,1,0,4,4,0, sum = 10 s². Unbiased estimate = 10/6 ≈ 1.67 s². (Using N−1 = 5 here would over-correct.)
(C) Prove that E[s] < σ in general, so that s underestimates the standard deviation even though s² is unbiased. (Assume Var(s) > 0.)
Solution
The square-root function is strictly concave on (0,∞). Jensen's inequality for a strictly concave g gives E[g(Y)] < g(E[Y]) whenever Y is non-degenerate. Take Y = s² and g = √·: E[s] = E[√s²] < √E[s²] = √σ² = σ. The inequality is strict because s² has nonzero variance. Hence s is biased low; the bias is of order σ/(4N) for large N and, under normality, is removed by the correction factor c₄(N).