Chebyshev's inequality

Statement

Let \((\Omega,\mathcal{F},\mathbb{P})\) be a probability space and let \(X:\Omega\to\mathbb{R}\) be a random variable with \(\mathbb{E}[X^2] \lt \infty\) (so that the mean \(\mu = \mathbb{E}[X]\) and the variance \(\sigma^2 = \operatorname{Var}(X) = \mathbb{E}[(X-\mu)^2]\) both exist and are finite). Then for every real \(k \gt 0\), \[ \mathbb{P}\bigl(|X-\mu| \geq k\bigr) \;\leq\; \frac{\sigma^2}{k^2}. \] Equivalently, in standardised form: for every \(t \gt 0\), \(\mathbb{P}(|X-\mu| \geq t\sigma) \leq 1/t^2\) (provided \(\sigma \gt 0\)).

Why it matters

Chebyshev's inequality is the first genuinely distribution-free concentration result a probabilist meets: it converts a single number you can often compute — the variance — into a quantitative bound on how likely a random variable is to stray from its mean. No knowledge of the distribution's shape, symmetry, or smoothness is required; only that a second moment exists.

Its most important consequence is the weak law of large numbers: applied to a sample mean, whose variance shrinks like \(1/n\), it forces the sample mean to concentrate near the true mean. Historically this is the route by which Chebyshev (1867) — and before him Bienaymé (1853) — first made the law of large numbers rigorous for general distributions, and it remains the template for the whole modern theory of concentration inequalities.

Hypotheses

Proof

The proof factors through Markov's inequality, which we prove in full rather than cite blind. Throughout, \(\mathbf{1}_A\) denotes the indicator function of the event \(A\).

Result

Corollaries & converses

• Standardised form. If \(\sigma \gt 0\), then \(\mathbb{P}(|X-\mu| \geq t\sigma) \leq 1/t^2\) for all \(t \gt 0\) (substitute \(k = t\sigma\)).
• Weak law of large numbers. If \(X_1, X_2, \ldots\) are i.i.d. with mean \(\mu\) and variance \(\sigma^2 \lt \infty\), then \(\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i\) satisfies \(\mathbb{P}(|\bar{X}_n - \mu| \geq \varepsilon) \leq \sigma^2/(n\varepsilon^2) \to 0\); i.e. \(\bar{X}_n \to \mu\) in probability. (Only pairwise uncorrelatedness is actually used.)
• Zero variance forces constancy. If \(\sigma^2 = 0\), take \(k = 1/n\) and let \(n \to \infty\): \(\mathbb{P}(|X-\mu| \gt 0) = \lim_n \mathbb{P}(|X-\mu| \geq 1/n) = 0\), so \(X = \mu\) almost surely.
• Converse fails (tightness). Small tails do not mean Chebyshev is accurate: for \(X \sim N(0,1)\), \(\mathbb{P}(|X| \geq 3) \approx 0.0027\), while Chebyshev gives only \(1/9 \approx 0.111\) — off by a factor of \(40\).
• Converse fails (moments). A tail bound of Chebyshev type does not imply finite variance: the density \(f(x) = |x|^{-3}\) for \(|x| \geq 1\) (suitably normalised) satisfies \(\mathbb{P}(|X| \geq k) = \mathcal{O}(1/k^2)\), yet \(\mathbb{E}[X^2] = \int_{|x|\geq 1} x^2 |x|^{-3}\,dx = \infty\). The borderline decay \(1/k^2\) is not summable against \(x^2\).

Fails without

Common errors

• Dividing by \(k\) instead of \(k^2\). \(\sigma^2/k\) is Markov applied to the wrong variable; the squaring in Step 2 is what produces \(k^2\).
• Confusing \(k\) with "number of standard deviations". \(\mathbb{P}(|X-\mu| \geq k) \leq \sigma^2/k^2\) has \(k\) in the original units; the \(1/t^2\) form requires \(k = t\sigma\). Mixing the two forms silently rescales the bound by \(\sigma^2\).
• Expecting a two-sided bound to serve one-sided needs efficiently. Chebyshev bounds \(\mathbb{P}(X - \mu \geq k)\) only via \(\mathbb{P}(|X-\mu| \geq k)\); the sharper one-sided bound is Cantelli's \(\sigma^2/(\sigma^2 + k^2)\) (Problem 5), and halving Chebyshev's bound for a one-sided tail is invalid without a symmetry assumption.
• Treating the bound as an approximation. \(\sigma^2/k^2\) is a worst-case ceiling attained only by extremal three-point distributions; using it as an estimate for, say, Gaussian tails overstates them enormously.
• Worrying about strict vs. non-strict inequality. Since \(\{|X-\mu| \gt k\} \subseteq \{|X-\mu| \geq k\}\), the same bound holds for the strict-inequality event by monotonicity of \(\mathbb{P}\); but students sometimes claim the reverse containment, which is false when \(\mathbb{P}(|X-\mu| = k) \gt 0\).
• Applying it when \(k \leq \sigma\) and reporting a "probability" larger than \(1\). The bound is correct but vacuous there; the honest statement is \(\mathbb{P} \leq \min(1, \sigma^2/k^2)\).

Discussion

Chebyshev's inequality is the archetype of a moment method: pass from an event to a non-negative random variable dominating its indicator, and take expectations. Markov's inequality is the raw form (first moment, one-sided variable); Chebyshev is Markov applied to \((X-\mu)^2\). The same template with \((X-\mu)^{2p}\) gives bounds of order \(\mathbb{E}[(X-\mu)^{2p}]/k^{2p}\), and with \(e^{\lambda(X-\mu)}\) it gives the Chernoff bound \(\mathbb{P}(X - \mu \geq k) \leq e^{-\lambda k}\,\mathbb{E}[e^{\lambda(X-\mu)}]\), whose optimisation over \(\lambda \gt 0\) yields exponential tail bounds (Hoeffding, Bernstein) when the moment generating function exists. Chebyshev is thus the second rung of an infinite ladder, and the weakest bound on it that is still strong enough to prove the weak law of large numbers.

Historically the inequality was published by Irénée-Jules Bienaymé in 1853 and rediscovered — with the application to the law of large numbers — by Pafnuty Chebyshev in 1867; the name "Bienaymé–Chebyshev inequality" is common in the French tradition. Chebyshev's motivation was to free the law of large numbers from the Bernoulli/de Moivre setting of binomial variables: his proof works for arbitrary uncorrelated summands with bounded variances, an enormous generalisation for its time. His students Markov and Lyapunov pushed the moment method further, to Markov's inequality in its modern form and to the central limit theorem respectively.

The inequality is sharp: for fixed \(\sigma^2\) and \(k\) with \(\sigma \leq k\), the distribution \(\mathbb{P}(X = \pm k) = \frac{\sigma^2}{2k^2}\), \(\mathbb{P}(X = 0) = 1 - \frac{\sigma^2}{k^2}\) has mean \(0\), variance exactly \(\sigma^2\), and achieves \(\mathbb{P}(|X| \geq k) = \sigma^2/k^2\) with equality. So no distribution-free improvement is possible. Improvements require extra structure: unimodality gives the Vysochanskij–Petunin inequality \(\mathbb{P}(|X-\mu| \geq t\sigma) \leq \frac{4}{9t^2}\) for \(t \gt \sqrt{8/3}\), and one-sided information gives Cantelli's inequality.

From a modern viewpoint, Chebyshev's inequality is a statement about the embedding \(L^2(\Omega) \subseteq L^{2,\infty}(\Omega)\): it says precisely that the weak-\(L^2\) quasinorm of \(X - \mu\) is controlled by its \(L^2\) norm, \(\sup_{k \gt 0} k^2\, \mathbb{P}(|X-\mu| \geq k) \leq \lVert X-\mu \rVert_{L^2}^2\). This is the probabilistic instance of the general Chebyshev–Markov inequality of measure theory, valid on any measure space, and it is the standard entry point to interpolation theory (Marcinkiewicz) and to the maximal-function estimates of harmonic analysis. The counterexample density \(|x|^{-3}\) above is exactly a function in weak-\(L^2\) but not in \(L^2\): the gap between the two spaces is the gap between the converse and the theorem.

Common misconceptions. Chebyshev does not say deviations of more than \(t\) standard deviations are "impossible" or even "rare" in an absolute sense — for \(t \leq 1\) it says nothing at all. It is not an approximation to the true tail probability, and it does not require (or become an equality under) normality; for a Gaussian it is very loose. Finally, the "\(68\)–\(95\)–\(99.7\)" rule is a Gaussian fact, not a consequence of Chebyshev, which guarantees only "\(0\)–\(75\)–\(88.9\)" in the same units.

Worked examples

Example 1 (coin tossing). A fair coin is tossed \(100\) times; let \(S\) be the number of heads. Bound the probability that \(S\) differs from \(50\) by at least \(10\).

Example 2 (sample-size design via the weak law). Measurements \(X_1, \ldots, X_n\) are i.i.d. with unknown mean \(\mu\) and known variance \(\sigma^2 = 1\). How large must \(n\) be so that the sample mean \(\bar{X}_n\) satisfies \(\mathbb{P}(|\bar{X}_n - \mu| \geq 0.1) \leq 0.05\)?

Problems

1. Let \(X \sim \mathrm{Unif}[0,1]\). Compute the Chebyshev bound on \(\mathbb{P}\bigl(|X - \frac{1}{2}| \geq \frac{1}{4}\bigr)\), compute the exact probability, and comment.
   Solution

   For the uniform distribution on \([0,1]\), \(\mu = \frac{1}{2}\) and \(\sigma^2 = \int_0^1 (x - \frac{1}{2})^2\,dx = \frac{1}{12}\). Chebyshev with \(k = \frac{1}{4}\) gives \[ \mathbb{P}\Bigl(\bigl|X - \tfrac{1}{2}\bigr| \geq \tfrac{1}{4}\Bigr) \;\leq\; \frac{1/12}{1/16} \;=\; \frac{4}{3}. \] The exact probability is \(\mathbb{P}(X \leq \frac{1}{4}) + \mathbb{P}(X \geq \frac{3}{4}) = \frac{1}{4} + \frac{1}{4} = \frac{1}{2}\). The bound \(\frac{4}{3} \gt 1\) is correct but vacuous: since here \(k = \frac{1}{4} \lt \sigma = \frac{1}{\sqrt{12}} \approx 0.289\), we are inside the regime \(k \leq \sigma\) where Chebyshev can say nothing. Moral: the inequality only bites for deviations exceeding one standard deviation.
2. Prove the standardised form: if \(\operatorname{Var}(X) = \sigma^2 \in (0, \infty)\), then \(\mathbb{P}(|X - \mu| \geq t\sigma) \leq \frac{1}{t^2}\) for all \(t \gt 0\). Deduce a bound on the probability that any random variable with finite variance lies more than \(5\) standard deviations from its mean.
   Solution

   Since \(\sigma \gt 0\) and \(t \gt 0\), the number \(k = t\sigma\) is strictly positive, so Chebyshev's inequality applies with this \(k\): \[ \mathbb{P}(|X - \mu| \geq t\sigma) \;\leq\; \frac{\sigma^2}{(t\sigma)^2} \;=\; \frac{\sigma^2}{t^2 \sigma^2} \;=\; \frac{1}{t^2}. \] With \(t = 5\): \(\mathbb{P}(|X - \mu| \geq 5\sigma) \leq \frac{1}{25} = 0.04\). Every distribution with finite variance — however heavy-tailed or skewed — puts at most \(4\%\) of its mass beyond \(5\) standard deviations. (Compare the Gaussian value \(\approx 5.7 \times 10^{-7}\): universality costs sharpness.)
3. Show that Chebyshev's inequality is sharp: for every \(\sigma^2 \gt 0\) and every \(k \geq \sigma\), construct a random variable with variance \(\sigma^2\) for which \(\mathbb{P}(|X - \mu| \geq k) = \frac{\sigma^2}{k^2}\) exactly.
   Solution

   Let \(p = \frac{\sigma^2}{2k^2}\); since \(k \geq \sigma \gt 0\) we have \(0 \lt p \leq \frac{1}{2}\), so we may define \[ \mathbb{P}(X = k) = \mathbb{P}(X = -k) = p, \qquad \mathbb{P}(X = 0) = 1 - 2p \geq 0. \] By symmetry \(\mu = \mathbb{E}[X] = k p + (-k)p + 0 = 0\). The variance is \[ \operatorname{Var}(X) = \mathbb{E}[X^2] = k^2 p + k^2 p = 2k^2 \cdot \frac{\sigma^2}{2k^2} = \sigma^2, \] as required. Finally \[ \mathbb{P}(|X - \mu| \geq k) = \mathbb{P}(X = k) + \mathbb{P}(X = -k) = 2p = \frac{\sigma^2}{k^2}, \] which is exactly the Chebyshev bound. Hence no inequality of the form \(\mathbb{P}(|X-\mu| \geq k) \leq c\,\sigma^2/k^2\) with \(c \lt 1\) can hold for all distributions: the constant \(1\) is optimal. (Tracing the proof, equality forces \((X-\mu)^2\) to be supported on \(\{0, k^2\}\) — precisely the three-point shape used here.)
4. (Weak law of large numbers.) Let \(X_1, X_2, \ldots\) be pairwise uncorrelated random variables with common mean \(\mu\) and variances \(\operatorname{Var}(X_i) \leq C\) for a uniform constant \(C\). Prove that \(\bar{X}_n = \frac{1}{n}\sum_{i=1}^n X_i \to \mu\) in probability, i.e. \(\mathbb{P}(|\bar{X}_n - \mu| \geq \varepsilon) \to 0\) for every \(\varepsilon \gt 0\).
   Solution

   Fix \(\varepsilon \gt 0\). By linearity, \(\mathbb{E}[\bar{X}_n] = \mu\). For pairwise uncorrelated summands the variance is additive — expanding, \(\operatorname{Var}\bigl(\sum_i X_i\bigr) = \sum_i \operatorname{Var}(X_i) + \sum_{i \neq j} \operatorname{Cov}(X_i, X_j)\) and every covariance term vanishes — so \[ \operatorname{Var}(\bar{X}_n) = \frac{1}{n^2} \sum_{i=1}^n \operatorname{Var}(X_i) \;\leq\; \frac{nC}{n^2} = \frac{C}{n}. \] In particular \(\bar{X}_n\) has finite variance, so Chebyshev's inequality applies with \(k = \varepsilon\): \[ \mathbb{P}\bigl(|\bar{X}_n - \mu| \geq \varepsilon\bigr) \;\leq\; \frac{\operatorname{Var}(\bar{X}_n)}{\varepsilon^2} \;\leq\; \frac{C}{n \varepsilon^2} \;\xrightarrow[n \to \infty]{}\; 0. \] Since \(\varepsilon \gt 0\) was arbitrary, \(\bar{X}_n \to \mu\) in probability. Note that full independence was never used — pairwise uncorrelatedness suffices, which is exactly the strength of the variance method.
5. (Cantelli's one-sided inequality.) Let \(X\) have mean \(\mu\) and variance \(\sigma^2 \lt \infty\). Prove that for every \(k \gt 0\), \( \mathbb{P}(X - \mu \geq k) \leq \frac{\sigma^2}{\sigma^2 + k^2}, \) and verify that this improves on the naive one-sided use of Chebyshev whenever \(k \gt \sigma\).
   Solution

   Write \(Z = X - \mu\), so \(\mathbb{E}[Z] = 0\) and \(\mathbb{E}[Z^2] = \sigma^2\). For any \(u \gt 0\), on the event \(\{Z \geq k\}\) we have \(Z + u \geq k + u \gt 0\), hence \[ \{Z \geq k\} \subseteq \{(Z+u)^2 \geq (k+u)^2\}, \] because \(x \mapsto x^2\) is increasing on \([0, \infty)\) and \(Z + u \geq k + u\) with both sides positive there. Applying Markov's inequality (Steps 3–5 of the main proof) to the non-negative variable \((Z+u)^2\) with threshold \((k+u)^2 \gt 0\): \[ \mathbb{P}(Z \geq k) \;\leq\; \frac{\mathbb{E}[(Z+u)^2]}{(k+u)^2} \;=\; \frac{\sigma^2 + 2u\,\mathbb{E}[Z] + u^2}{(k+u)^2} \;=\; \frac{\sigma^2 + u^2}{(k+u)^2}, \] using linearity of expectation and \(\mathbb{E}[Z] = 0\). This holds for every \(u \gt 0\); we minimise the right-hand side. Setting \(g(u) = \frac{\sigma^2 + u^2}{(k+u)^2}\), \[ g'(u) = \frac{2u(k+u)^2 - 2(k+u)(\sigma^2 + u^2)}{(k+u)^4} = \frac{2\bigl(uk - \sigma^2\bigr)}{(k+u)^3}, \] which vanishes at \(u^* = \sigma^2/k\), is negative before and positive after, so \(u^*\) is the global minimiser on \((0,\infty)\). Substituting \(u^* = \sigma^2/k\): \[ g(u^*) = \frac{\sigma^2 + \sigma^4/k^2}{(k + \sigma^2/k)^2} = \frac{\sigma^2 (k^2 + \sigma^2)/k^2}{(k^2 + \sigma^2)^2 / k^2} = \frac{\sigma^2}{\sigma^2 + k^2}. \] Hence \(\mathbb{P}(X - \mu \geq k) \leq \frac{\sigma^2}{\sigma^2 + k^2}\). Comparison: the naive one-sided use of Chebyshev is \(\mathbb{P}(X - \mu \geq k) \leq \mathbb{P}(|X - \mu| \geq k) \leq \frac{\sigma^2}{k^2}\), and \[ \frac{\sigma^2}{\sigma^2 + k^2} \lt \frac{\sigma^2}{k^2} \iff k^2 \lt \sigma^2 + k^2, \] which is always true — Cantelli is strictly better for every \(k \gt 0\), and unlike \(\sigma^2/k^2\) it is a genuine probability bound (below \(1\)) for all \(k \gt 0\). At \(k = \sigma\) it gives \(\frac{1}{2}\) where Chebyshev gives the vacuous \(1\).