The Inverse-Variance Weighted Mean

Derivation

The Inverse-Variance Weighted Mean

D-114 Home PU-106 Threads chance Depends on First-Order Propagation of Uncertainty, The Standard Error of the Mean

Statement

Given N independent, unbiased measurements x_i of a single quantity μ, each with variance σ_i², the linear combination x̂ = Σ_i w_i x_i that is unbiased and has the smallest possible variance is the one whose weights satisfy w_i ∝ 1/σ_i². The resulting minimum variance is 1/σ² = Σ_i 1/σ_i².

Why it matters

Almost every experimental number that carries an error bar is really several numbers averaged together: repeated runs, independent apparatus, or results from different laboratories. The naive equal-weight mean throws away information — it treats a crude reading and a precise one as equals. The inverse-variance weighted mean is the provably optimal way to fuse them, and it is the estimator that underlies world-average constants (particle masses, the gravitational constant, cosmological parameters).

The companion result, that inverse variances (precisions) simply add, is the quantitative statement of "more data helps." It tells you exactly how much a new measurement sharpens your knowledge, and warns you that a sufficiently imprecise measurement contributes essentially nothing.

Assumptions

The measurements are unbiased:each has expectation μ. If dropped, minimising variance no longer minimises total error — a precise-but-biased value would be over-weighted and drag the estimate off the true value. The measurements are statistically independent:all covariances Cov(x_i, x_j) = 0 for i ≠ j. If dropped, the variance is no longer a diagonal sum and the optimal weights involve the full inverse covariance matrix. Each variance σ_i² is known and finite:weights use the true variances. If dropped (variances estimated from the same data), the weights become random and correlated with the values, introducing bias (Peelle's Pertinent Puzzle).

The estimator is restricted to be linear in the data:we optimise only over linear combinations. If dropped, for non-Gaussian noise a nonlinear estimator can beat the linear one; for Gaussian noise the linear optimum is globally optimal, so this restriction costs nothing.

Derivation

x̂ = Σ_i w_i x_i

Restrict attention to linear combinations of the data; the w_i are constants to be chosen. A

⟨x̂⟩ = Σ_i w_i ⟨x_i⟩ = μ Σ_i w_i ⇒ Σ_i w_i = 1

Take the expectation and use ⟨x_i⟩ = μ. Demanding ⟨x̂⟩ = μ for every μ forces the weights to sum to one (the unbiasedness constraint). A

V ≡ Var(x̂) = Σ_i w_i² σ_i²

First-order error propagation for a linear function; independence kills every cross term 2 w_i w_j Cov(x_i, x_j), leaving only the diagonal. B

L(w, λ) = Σ_i w_i² σ_i² − λ( Σ_i w_i − 1 )

Minimise V subject to the constraint from step 2 using a Lagrange multiplier λ. B

∂L/∂w_i = 2 w_i σ_i² − λ = 0 ⇒ w_i = λ / (2σ_i²)

Stationarity: differentiate with respect to each w_i and set to zero. Every weight is inversely proportional to its variance — the central result already appears here. B

Σ_i w_i = (λ/2) Σ_i 1/σ_i² = 1 ⇒ λ/2 = 1 / S, S ≡ Σ_i 1/σ_i²

Fix the multiplier by imposing the constraint; define the total precision S to keep the algebra compact. B

w_i = (1/σ_i²) / S = (1/σ_i²) / Σ_j (1/σ_j²)

Substitute λ back. The normalised inverse-variance weights, guaranteed to sum to one. A

V_min = Σ_i w_i² σ_i² = Σ_i (1/σ_i⁴)/S² · σ_i² = (1/S²) Σ_i 1/σ_i² = S/S² = 1/S

Insert the optimal weights into the variance of step 3; the sum collapses to S, so the minimum variance is exactly 1/S. B

∂²L / ∂w_i ∂w_j = 2σ_i² δ_ij ≻ 0

The Hessian is diagonal with strictly positive entries, hence positive definite, so the stationary point of step 5 is a genuine minimum (not a saddle or maximum) on the constraint surface. C

Result

w_i = (1/σ_i²) / Σ_j(1/σ_j²) ⇒ x̂ = Σ_i(x_i/σ_i²) / Σ_i(1/σ_i²), 1/σ² = Σ_i 1/σ_i²

Reading. Weight each measurement by its precision, the reciprocal of its variance, then renormalise. The combined estimate's precision is the sum of the individual precisions — informations add, so the combined uncertainty is always smaller than that of the best single input.

Units check. The weights w_i are ratios of quantities with units [x]⁻², hence dimensionless, and Σ w_i = 1 as required; so x̂ carries the units of x. In 1/σ² = Σ 1/σ_i² every term has units [x]⁻², so the reciprocal σ² has units [x]², a proper variance.

Limiting cases

Equal variances σ_i = σ: all weights become 1/N, recovering the ordinary mean, and σ² = σ²/N — the standard error of the mean.
One dominant measurement σ_k → 0: its weight → 1, all others → 0; the estimate becomes x_k and σ → 0. A perfect measurement supersedes the rest.
One useless measurement σ_k → ∞: its weight → 0 and it drops out; adding an infinitely noisy datum leaves the estimate and its variance unchanged. A bad measurement can never hurt.
Two measurements: x̂ = (x₁/σ₁² + x₂/σ₂²)/(1/σ₁² + 1/σ₂²), with 1/σ² = 1/σ₁² + 1/σ₂² — the variances combine like resistors in parallel.

Breaks when

The measurements are correlated. If Cov(x_i,x_j) ≠ 0 (shared calibration, common systematic), the diagonal variance sum of step 3 is wrong. The optimal weights become w ∝ C⁻¹ 1 using the full covariance matrix C; naive inverse-variance weighting then underestimates the combined uncertainty, sometimes badly.
The measurements are biased. Minimising variance is not the same as minimising error when ⟨x_i⟩ ≠ μ. The mean-squared error is variance + bias²; inverse-variance weighting ignores the bias term and will heavily weight a precise-but-offset value, converging confidently on the wrong answer.
The variances are estimated from the data themselves. Then the weights are random and correlated with the x_i, producing a biased combined estimate (Peelle's Pertinent Puzzle), most severe when the quoted error scales with the measured value.
The noise is heavy-tailed / non-Gaussian. The minimum-variance linear estimator is no longer efficient; a few outliers dominate the weighted sum, and a robust estimator (median, trimmed mean) outperforms it.

Failure modes

Weighting by 1/σ instead of 1/σ². Confusing precision with the reciprocal of the standard deviation; the true weight is the reciprocal of the variance.
Forgetting to normalise. Using raw weights w_i = 1/σ_i² without dividing by S, so Σ w_i ≠ 1 and the estimate is scaled by S.
Averaging the uncertainties. Reporting the combined σ as the mean of the σ_i, rather than combining precisions via 1/σ² = Σ 1/σ_i².
Combined error larger than the best input. A sanity-check violation: the weighted-mean uncertainty must be ≤ the smallest σ_i; getting a larger value signals an arithmetic slip.
Using the scatter of the values as the uncertainty. Substituting the sample standard deviation of the x_i for the propagated σ_i; these answer different questions and coincide only under special conditions.

Discussion

The inverse-variance weighted mean is the best linear unbiased estimator (BLUE) of a constant from independent measurements — a special case of the Gauss–Markov theorem. "Best" here means minimum variance among all unbiased linear estimators, which is exactly the optimisation we carried out with a single Lagrange multiplier. The elegance is that the answer depends only on the second moments σ_i², not on the detailed shape of the noise.

The result 1/σ² = Σ 1/σ_i² is best read as a statement about information. For a Gaussian measurement of a mean, the Fisher information is precisely 1/σ_i²; because independent experiments carry additive Fisher information, the total information is the sum, and the Cramér–Rao bound sets the variance floor at its reciprocal. The weighted mean does not merely combine numbers well — it saturates the fundamental bound, extracting every bit of information the data contain.

The same estimator drops out of maximum likelihood. For Gaussian errors the negative log-likelihood is ½ Σ (x_i − μ)²/σ_i² (a chi-square); setting its derivative with respect to μ to zero reproduces x̂ exactly. So the least-squares fit, the maximum-likelihood estimate, and the minimum-variance linear estimate all coincide here — a reassuring convergence of three different optimality criteria.

The parallel-resistor analogy is more than a mnemonic. Precisions 1/σ_i² add exactly as conductances do, and the combined system is always "stiffer" (more precise) than any component. This additive structure generalises: in a Kalman filter each measurement update adds its precision to the running estimate, and the posterior precision of a Gaussian is the sum of prior and likelihood precisions — Bayesian updating is inverse-variance weighting between a prior and a datum. Common misconceptions: that the weighted mean must lie between the individual values (true here, but only because all weights are positive and sum to one); and that combining many mediocre measurements can beat one excellent one — it can, but only if their errors are genuinely independent, which shared systematics quietly destroy.

Worked examples

Two length measurements: x₁ = 5.02 ± 0.04 m, x₂ = 5.10 ± 0.02 m. Find the weighted mean and its uncertainty.

Symbolic form first: x̂ = (x₁/σ₁² + x₂/σ₂²)/S, S = 1/σ₁² + 1/σ₂². A

1/σ₁² = 1/(0.04)² = 625 m⁻², 1/σ₂² = 1/(0.02)² = 2500 m⁻²

Compute the two precisions; the more precise value carries four times the weight. A

S = 625 + 2500 = 3125 m⁻²

Total precision. A

x̂ = (5.02·625 + 5.10·2500)/3125 = (3137.5 + 12750)/3125 = 15887.5/3125 = 5.084 m

Weighted average; the result is pulled toward the more precise x₂. A

σ = 1/√S = 1/√3125 = 0.0179 m

Combined uncertainty — smaller than either input, as it must be. A

x̂ = 5.084 ± 0.018 m

Reading. The estimate sits closer to the sharper measurement, and the uncertainty (0.018 m) beats the best single input (0.02 m).

Units check. Precisions in m⁻², their reciprocal square-root in m — a length uncertainty, as required.

Three determinations of g: 9.79 ± 0.05, 9.82 ± 0.03, 9.85 ± 0.04 m/s². Combine them.

General form x̂ = Σ(x_i/σ_i²)/S with S = Σ 1/σ_i². A

1/σ_i² = 400, 1111.1, 625 (s²/m²)

From 1/0.05², 1/0.03², 1/0.04² respectively. A

S = 400 + 1111.1 + 625 = 2136.1 (s²/m²)

Total precision. A

x̂ = (9.79·400 + 9.82·1111.1 + 9.85·625)/2136.1 = 20983.4/2136.1 = 9.823 m/s²

Numerator: 3916 + 10911.1 + 6156.25. The mid value dominates because it is most precise. A

σ = 1/√2136.1 = 0.0216 m/s²

Combined uncertainty. A

g = 9.823 ± 0.022 m/s²

Reading. Fusing three modest determinations yields an uncertainty (0.022) below the best single one (0.03), because precisions add.

Units check. 1/σ_i² in s² m⁻²; reciprocal root gives m s⁻², an acceleration.

Problems

Two measurements with equal errors: x₁ = 10.0 ± 0.5 and x₂ = 11.0 ± 0.5. Show the weighted mean reduces to the ordinary mean, and find the combined uncertainty.
Solution
Equal σ means equal weights w₁ = w₂ = ½, so x̂ = (10.0 + 11.0)/2 = 10.5. Precisions 1/0.25 = 4 each, S = 8, σ = 1/√8 = 0.354 = σ/√2. Result: 10.50 ± 0.35, matching the standard-error-of-the-mean scaling.
Combine x₁ = 100.2 ± 0.3 with x₂ = 100.5 ± 0.1.
Solution
1/σ₁² = 1/0.09 = 11.11, 1/σ₂² = 1/0.01 = 100, S = 111.11. x̂ = (100.2·11.11 + 100.5·100)/111.11 = (1113.3 + 10050)/111.11 = 100.47. σ = 1/√111.11 = 0.095. Result: 100.47 ± 0.09, strongly pulled toward the nine-times-more-precise x₂.
To x₁ = 5.00 ± 0.10 a colleague adds a very crude value x₂ = 4.00 ± 10.0. Show the crude value barely changes the answer.
Solution
1/σ₁² = 100, 1/σ₂² = 1/100 = 0.01, S = 100.01. x̂ = (5.00·100 + 4.00·0.01)/100.01 = (500 + 0.04)/100.01 = 4.9999, and σ = 1/√100.01 = 0.09999. The estimate shifts by ≈ 10⁻⁴ and the uncertainty by ≈ 5×10⁻⁶: 5.000 ± 0.100, essentially unchanged — a nearly useless measurement contributes almost nothing.
You have N independent measurements each with the same uncertainty σ. Show the weighted mean's uncertainty is σ/√N, and find how many are needed to reach σ/10.
Solution
Equal variances give S = N/σ², so the combined variance is 1/S = σ²/N and σ_comb = σ/√N — exactly the standard error of the mean. Setting σ/√N = σ/10 gives √N = 10, i.e. N = 100. Tenfold precision costs a hundredfold more data — the familiar diminishing return of independent averaging.
Starting from the Gaussian log-likelihood, prove the inverse-variance weighted mean is the maximum-likelihood estimator of μ from independent Gaussian measurements x_i ∼ N(μ, σ_i²).
Solution
The negative log-likelihood (dropping constants) is χ²(μ) = ½ Σ_i (x_i − μ)²/σ_i². Differentiate and set to zero: dχ²/dμ = − Σ_i (x_i − μ)/σ_i² = 0, so Σ_i x_i/σ_i² = μ Σ_i 1/σ_i², giving μ̂ = Σ(x_i/σ_i²)/Σ(1/σ_i²) — the inverse-variance weighted mean. The second derivative d²χ²/dμ² = Σ 1/σ_i² = S > 0 confirms a minimum of χ² (maximum of likelihood), and its reciprocal 1/S is the estimator's variance — matching the minimum-variance result and saturating the Cramér–Rao bound.