The Chi-Squared Distribution
~6 min read · Tier toggle changes depth (T1 plain · T2 standard · T3 rigour)
You should be comfortable with the properties of the standard normal distribution, the definition of the Gamma function, and the concept of independent random variables.
Why this matters
The chi-squared distribution is the mathematical backbone of hypothesis testing and confidence intervals in experimental physics.
In the previous lecture, we established the Cramér-Rao bound, which tells us the fundamental limit on the precision of any unbiased estimator. To determine whether our experimental data actually achieves this limit, or to test if a theoretical model is consistent with our observations, we need a distribution that describes the sum of squared deviations. The chi-squared distribution provides exactly this. It arises naturally when we square standard normal variables, making it the essential tool for quantifying "goodness of fit" and for constructing confidence regions in multi-parameter fits.
Intuition: Summing Squared Deviations
Imagine you are measuring a physical quantity, such as the length of a rod, multiple times. If your instrument is well-calibrated and the only errors are random, your measurements will scatter around the true value. If you subtract the true value from each measurement and divide by the standard deviation, you get a set of numbers that follow a standard normal distribution. These numbers can be positive or negative. However, if you square these numbers, they all become positive. The sum of these squared, normalized deviations is a measure of how "spread out" your data is relative to the expected noise level. This sum is not a fixed number; it fluctuates from one experiment to another. The chi-squared distribution describes the probability of obtaining a specific value for this sum. It is a distribution of positive values only, skewed to the right, and its shape depends on how many independent measurements (or degrees of freedom) you have summed together.
Definition and Probability Density
Formally, let \( Z_1, Z_2, \dots, Z_k \) be independent random variables, each following the standard normal distribution \( N(0, 1) \). We define the random variable \( Q \) as the sum of their squares:
The variable \( Q \) is said to follow the chi-squared distribution with \( k \) degrees of freedom, denoted as \( \chi^2_k \). The probability density function (PDF) of \( Q \) is given by:
where \( \Gamma(\cdot) \) is the Gamma function. For integer \( k \), \( \Gamma(k/2) \) simplifies to factorials or half-factorials, making the expression more concrete. The support of this distribution is strictly positive, reflecting the fact that a sum of squares cannot be negative. As \( k \) increases, the distribution becomes more symmetric and bell-shaped, centered around its mean.
Moments and Variance
To understand the behavior of \( Q \), we examine its first two moments. The expected value (mean) of a chi-squared variable with \( k \) degrees of freedom is equal to \( k \):
The variance, which measures the spread of the distribution, is twice the number of degrees of freedom:
These results are crucial for experimental analysis. If you calculate a chi-squared statistic from your data, you expect it to be close to \( k \). If it is significantly larger, it suggests that your model is a poor fit or that your error estimates are too small. If it is significantly smaller, it may indicate that your error estimates are too large or that the model is over-parameterized. The standard deviation of the chi-squared statistic is \( \sqrt{2k} \), which grows with the square root of the degrees of freedom.
Connection to the Gamma Distribution
The chi-squared distribution is a special case of the Gamma distribution. A random variable \( X \) follows a Gamma distribution with shape parameter \( \alpha \) and scale parameter \( \theta \) if its PDF is proportional to \( x^{\alpha-1} e^{-x/\theta} \). Comparing this with the chi-squared PDF, we can identify the parameters as:
This relationship allows us to use the general properties of the Gamma distribution to derive the moments of the chi-squared distribution. Specifically, the mean of a Gamma distribution is \( \alpha \theta \) and the variance is \( \alpha \theta^2 \). Substituting the chi-squared parameters, we recover \( E[Q] = \frac{k}{2} \cdot 2 = k \) and \( \mathrm{Var}(Q) = \frac{k}{2} \cdot 2^2 = 2k \). This connection is useful because many statistical software packages and libraries implement the Gamma distribution directly, allowing for efficient computation of chi-squared probabilities.
The derivation
The Chi-Squared Distribution
Let Z1, …, Zk be independent standard-normal variables, Zi ~ N(0, 1), and define Q = Σi=1k Zi2. Then Q follows the chi-squared distribution with k degrees of freedom, with probability density fQ(x) = xk/2 − 1 e−x/2 ⁄ [2k/2 Γ(k/2)] for x > 0, mean E[Q] = k, and variance Var(Q) = 2k.
A common pitfall is confusing the number of data points with the degrees of freedom. If you fit a model with \( p \) free parameters to \( N \) data points, the degrees of freedom for the chi-squared test are \( k = N - p \), not \( N \). Using \( N \) instead of \( N - p \) will lead to incorrect p-values and confidence intervals, as the expected value of the statistic is \( N - p \), not \( N \). C
Worked example
Problem. A physicist measures the decay time of a radioactive isotope 10 times. The theoretical model predicts a mean decay time of 5.0 s with a standard deviation of 1.0 s. The measured values are: 5.1, 4.9, 5.2, 4.8, 5.0, 5.1, 4.9, 5.0, 5.1, 4.9 s. Calculate the chi-squared statistic and determine if the model is consistent with the data at the 95% confidence level. Assume the critical value for \( \chi^2_{10} \) at 95% is 18.31 and for \( \chi^2_{9} \) is 16.92.
- The model fixes both the mean, \( \mu = 5.0\ \mathrm{s} \), and the standard deviation, \( \sigma = 1.0\ \mathrm{s} \); nothing is fitted to these data, so the number of degrees of freedom is \( k = N = 10 \). (If the mean had been estimated from the same data, it would be \( k = 9 \).)
- Residuals \( x_i - \mu \): \( 0.1, -0.1, 0.2, -0.2, 0, 0.1, -0.1, 0, 0.1, -0.1\ \mathrm{s} \).
- Divide each squared residual by \( \sigma^2 = 1.0\ \mathrm{s^2} \) and add: \( \chi^2 = 0.01 + 0.01 + 0.04 + 0.04 + 0 + 0.01 + 0.01 + 0 + 0.01 + 0.01 = 0.14 \).
- Compare with the 95% critical value for \( k = 10 \): \( 0.14 < 18.31 \), so the model is not rejected.
- But \( \chi^2 \) is far below its expected value \( k = 10 \): the probability of \( \chi^2_{10} \le 0.14 \) is only about \( 10^{-8} \). The data scatter with a sample standard deviation of about \( 0.12\ \mathrm{s} \), not \( 1.0\ \mathrm{s} \), so the stated uncertainty is far too large (or the data are not independent).
Units check. Each term \( (x_i - \mu)^2/\sigma^2 \) is \( \mathrm{s^2}/\mathrm{s^2} \), so \( \chi^2 \) is dimensionless, as it must be to be compared with a \( \chi^2_k \) table.
Three quick tests
- What is the mean of a chi-squared distribution with 5 degrees of freedom?
Answer
The mean is equal to the degrees of freedom, so the mean is 5. - If you have 20 data points and fit a model with 2 free parameters, what is the degrees of freedom for the chi-squared test?
Answer
The degrees of freedom are \( N - p = 20 - 2 = 18 \). - Is the chi-squared distribution symmetric?
Answer
No, it is skewed to the right, especially for small degrees of freedom. It becomes more symmetric as the degrees of freedom increase.
Problem set for PU-106 →
Each derivation in this unit ends with five graded problems with full solutions.
Lecture notes drafted with a local model and checked line by line by the editor; every number in the worked example was recomputed.
Derivations cited: D-123 · Threads: chance