Convergence of fixed-point iteration
Statement
Let \((X,d)\) be a non-empty complete metric space and let \(g:X\to X\) be a contraction, i.e. there exists a constant \(L\) with \(0 \le L \lt 1\) such that \(d(g(x),g(y)) \le L\,d(x,y)\) for all \(x,y \in X\). Then: (i) \(g\) has a unique fixed point \(x^\ast \in X\) with \(g(x^\ast)=x^\ast\); (ii) for any starting point \(x_0 \in X\), the iteration \(x_{n+1} = g(x_n)\) converges to \(x^\ast\), i.e. \(x_n \to x^\ast\) as \(n \to \infty\); (iii) the convergence is at least linear (geometric), with the a priori bound \(d(x_n,x^\ast) \le \dfrac{L^n}{1-L}\,d(x_1,x_0)\) and the a posteriori bound \(d(x_n,x^\ast) \le \dfrac{L}{1-L}\,d(x_n,x_{n-1})\), for all \(n \ge 1\).
Why it matters
This is the Banach fixed-point theorem, specialised to the setting most useful in numerical analysis: it is the single result that certifies, in advance and without knowing the answer, that an iterative scheme \(x_{n+1}=g(x_n)\) will converge, and it hands you an explicit, computable error bound as it does so. Almost every fixed-point-style solver used in practice — Newton's method (locally, via its own contraction constant), the Jacobi and Gauss–Seidel iterations for linear systems written as \(x = Bx+c\), Picard iteration for ODE existence, and fixed-point schemes for nonlinear equations \(x=g(x)\) — is, at bottom, an appeal to this theorem or a close relative of it.
What makes the result genuinely useful rather than merely elegant is that it converts a qualitative existence statement into a quantitative stopping criterion: the a posteriori bound lets you certify, from two consecutive computed iterates alone, how close you already are to the true root, which is exactly what a numerical routine needs to decide when to stop.
Hypotheses
Proof
Result
Reading. If repeatedly applying a map always shrinks distances by at least a fixed factor \(L\lt1\), and the space has no "holes" for a limit to fall through, then repeated application from any starting guess homes in on one and only one fixed point, and it does so at a guaranteed geometric rate you can bound without ever knowing the fixed point itself.
Scope. Applies to any complete metric space — finite-dimensional \(\mathbb{R}^n\) with any equivalent norm, Banach spaces of functions (giving Picard–Lindelöf existence for ODEs), or discrete iteration schemes in numerics. It does not require differentiability of \(g\), linearity, or that \(X\) be a vector space — only a metric and completeness. It gives global convergence from any \(x_0\in X\); this is stronger than, and should not be confused with, purely local convergence results such as Newton's method's quadratic convergence, which only apply once \(x_0\) is already close to \(x^\ast\).
Corollaries & converses
- Rate corollary. The convergence is at least linear with asymptotic error constant \(L\): from Step 9, \(\limsup_{n\to\infty} d(x_n,x^\ast)^{1/n} \le L\), so smaller \(L\) means faster guaranteed convergence.
- Stopping-criterion corollary. The a posteriori bound (Step 10) is used directly in practice: once \(\frac{L}{1-L}d(x_n,x_{n-1}) \lt \text{tol}\), you may certify \(d(x_n,x^\ast)\lt\text{tol}\) without ever computing \(x^\ast\).
- Differentiable corollary (mean value form). If \(X\subseteq\mathbb{R}\) is a closed interval and \(g\) is differentiable with \(\sup_{x\in X}|g'(x)| = L \lt 1\), then \(g\) is a contraction with that constant \(L\), by the Mean Value Theorem applied to \(g\) on the segment between any \(x,y\in X\).
- Converse fails. A map can have a unique, attracting fixed point that iteration converges to, without being a contraction in any metric compatible with the topology on a natural domain — e.g. \(g(x)=x^2\) on \([0,1)\) has fixed point \(0\) attracting all points, but \(g\) is not a contraction on all of \([0,1)\) since \(|g(x)-g(0)|/|x-0| = x \to 1\) as \(x\to1\), and completeness of \([0,1)\) also fails. So convergence of an iteration does not imply the hypotheses of this theorem held; the theorem gives a sufficient, not necessary, condition.
- Local converse partly holds. If \(g\) is \(C^1\) near a fixed point \(x^\ast\) with \(|g'(x^\ast)|\lt1\), then \(g\) is a contraction on some sufficiently small closed neighbourhood of \(x^\ast\) (by continuity of \(g'\)), recovering local applicability of the theorem even when \(g\) is not a global contraction.
Fails without
- Drop \(L\lt1\) (allow \(L=1\), i.e. only \(d(g(x),g(y))\le d(x,y)\)): take \(X=\mathbb{R}\), \(g(x)=x+1\). Then \(d(g(x),g(y))=d(x,y)\) exactly (an isometry, so the weak inequality holds with \(L=1\)), \(g\) has no fixed point at all, and the iterates \(x_n = x_0+n \to \infty\), diverging.
- Drop completeness of \(X\): take \(X=(0,1]\) (not complete, missing its limit point \(0\)) with \(g(x)=x/2\), which satisfies \(d(g(x),g(y)) = \frac12 d(x,y)\), a genuine contraction with \(L=1/2\). The iterates \(x_n = x_0/2^n \to 0\), but \(0\notin X\), so the "fixed point" the sequence is converging to does not exist inside the space — existence of \(x^\ast\in X\) fails even though the Cauchy estimate (Steps 1–5) is unaffected.
- Drop self-mapping \(g:X\to X\): on \(X=[0,1]\), let \(g(x) = 2x \bmod{\text{(defined only for }x\le1/2)}\), i.e. \(g\) is only defined on \([0,1/2]\) but \(x_0=0.9\) is not in the domain at all — the iteration is not even well-posed, so "convergence" is meaningless before contraction is ever invoked.
Common errors
- Checking \(|g(x)-g(y)| \lt |x-y|\) at isolated test points (or even "for all \(x\ne y\) pointwise") and treating that as verifying a contraction, instead of bounding \(\sup |g'|\) or the Lipschitz ratio uniformly over the whole domain — this is exactly the trap in the \(g(x)=x+1/x\) counterexample above.
- Using the a priori error bound with the wrong base distance, e.g. writing \(\frac{L^n}{1-L}d(x_n,x_{n-1})\) instead of \(d(x_1,x_0)\) — conflating the a priori bound (Step 9, uses the very first step) with the a posteriori bound (Step 10, uses the latest step).
- Forgetting to verify \(g\) maps the candidate domain into itself before checking the contraction ratio, so the iteration silently leaves the region where the Lipschitz bound was established.
- Assuming any \(L \lt 1\) found on a subinterval automatically applies globally, when \(g'\) may exceed 1 elsewhere on the full stated domain \(X\).
- Confusing this global, any-starting-point convergence with the local quadratic convergence of Newton's method, then being surprised that a "fixed-point" reformulation of Newton's method converges only linearly and only near the root unless the contraction hypothesis is separately re-verified for it.
Discussion
The result is usually named for Stefan Banach, who proved it in his 1922 thesis in the abstract setting of complete normed (later: metric) spaces, but the underlying iterative idea — successive approximation — long predates that abstraction, appearing in Picard's 1890 existence proof for ordinary differential equations and, in disguised numerical form, in far older root-finding recipes. Banach's contribution was to isolate exactly the right hypotheses (completeness plus a uniform contraction ratio) so that the same three-line argument proves existence, uniqueness, and a constructive convergence rate all at once — a rare combination in existence theorems, most of which give you no algorithm.
In numerical analysis the theorem is less often invoked by its full metric-space name and more often used through its \(\mathbb{R}^n\) or \(C^1\) corollary: to solve \(f(x)=0\), one rewrites it as \(x=g(x)\) for some cleverly chosen \(g\) (there are infinitely many rewritings of the same root equation), and the entire practical question is choosing \(g\) so that \(|g'(x^\ast)|\lt1\) near the root — this is precisely why naive fixed-point iteration on \(x=\cos x\) rewritten badly can diverge while a rescaled version converges rapidly: the algebra of the equation is unchanged, only the contraction constant of the chosen \(g\) changes.
There is a genuine conceptual link to Newton's method: Newton iteration for \(f\) is fixed-point iteration for \(g(x) = x - f(x)/f'(x)\), and one computes \(g'(x^\ast) = f(x^\ast)f''(x^\ast)/f'(x^\ast)^2 = 0\) whenever \(x^\ast\) is a simple root, which is exactly why Newton's method achieves quadratic (not merely the linear rate guaranteed by this theorem) convergence locally — this theorem only certifies that Newton's method converges at all once you are inside a suitable ball, not the faster rate, which needs a separate, sharper Taylor-remainder argument.
Common misconception: that "the map looks like it's shrinking distances" is enough — informally checking a few sample points, or even checking \(|g'|\lt1\) at the fixed point alone (rather than uniformly on the whole relevant domain, or at least on a closed neighbourhood as in the local converse corollary), is not a valid verification of the hypothesis and is the single most common source of false claims of convergence in student work.
Worked examples
Problems
- Let \(X=[1,2]\) and \(g(x) = \frac{1}{2}\left(x + \frac{2}{x}\right)\). Show \(g:X\to X\) and that \(g\) is a contraction on \(X\); find the smallest \(L\) your argument gives.
Solution
\(g'(x) = \frac12 - \frac{1}{x^2}\). On \([1,2]\), \(g'(1)=\frac12-1=-\frac12\) and \(g'(2)=\frac12-\frac14=\frac14\); since \(g'\) is increasing on \([1,2]\) (as \(g''(x)=2/x^3\gt0\)), \(g'\) ranges continuously over \([-\frac12,\frac14]\), so \(|g'(x)|\le\frac12\) for all \(x\in[1,2]\). By the Mean Value Theorem corollary, \(g\) is a contraction with \(L=\frac12\). For self-mapping: \(g\) has a minimum where \(g'(x)=0\), i.e. \(x=\sqrt2\), giving \(g(\sqrt2)=\sqrt2\approx1.414\); endpoints give \(g(1)=1.5\), \(g(2)=1.5\); since \(g\) is decreasing then increasing on \([1,2]\) with minimum \(\sqrt2\gt1\) and both endpoint values \(1.5\le2\), we get \(g(X)\subseteq[\sqrt2,1.5]\subset[1,2]=X\). Both hypotheses hold with \(L=\frac12\). (This is Newton's method for \(\sqrt2\), and indeed \(x^\ast=\sqrt2\).) - For the map in Problem 1, starting from \(x_0=2\), compute \(x_1,x_2\) and use the a priori bound to guarantee \(d(x_4,x^\ast)\) is below what tolerance?
Solution
\(x_1 = \frac12(2+1)=1.5\). \(x_2=\frac12(1.5+2/1.5)=\frac12(1.5+1.3333)=1.41667\). A priori bound: \(d(x_n,x^\ast)\le \frac{L^n}{1-L}d(x_1,x_0)\), with \(L=\frac12\), \(d(x_1,x_0)=|1.5-2|=0.5\). For \(n=4\): \(\frac{(1/2)^4}{1/2}\times0.5 = \frac{1/16}{1/2}\times0.5 = \frac18\times0.5=0.0625\). So \(d(x_4,x^\ast)\le0.0625\) is guaranteed (in fact actual convergence is much faster here since this is Newton's method with quadratic local rate, but the theorem only certifies the linear bound). - Show that \(g(x) = 4x(1-x)\) (the logistic map at parameter 4) is not a contraction on \(X=[0,1]\), and explain why this does not by itself prove the iteration diverges.
Solution
\(g'(x)=4-8x\), so \(|g'(x)|\) ranges up to \(|g'(0)|=4\) and \(|g'(1)|=4\), far exceeding 1; more directly, take \(x=0.01,\,y=0\): \(g(x)-g(y) = 4(0.01)(0.99) - 0 = 0.0396\), while \(x-y=0.01\), giving ratio \(3.96\), and this ratio is unbounded as points near \(x=0,1\) are chosen closer together with one fixed at an endpoint... more carefully, near \(x=1/2\) also \(g'(1/2)=0\), so the ratio is not uniformly bounded below 1 anywhere on \([0,1]\); hence \(g\) fails the contraction hypothesis on all of \(X\). This does not prove divergence because the theorem only gives a sufficient condition for convergence (see Corollaries: converse fails); indeed the logistic map at parameter 4 is a classical example of chaotic, non-convergent behaviour for generic starting points, consistent with the hypothesis failing, but the failure of the hypothesis alone is not a proof of chaos — that requires separate argument (e.g. conjugacy to the tent map). - Let \(X=\mathbb{R}^2\) with the Euclidean norm and \(g(x) = Bx\) where \(B\) is a \(2\times2\) matrix. State a necessary and sufficient condition on \(B\) for \(g\) to be a contraction on all of \(X\) in the \(\ell^2\) operator norm, and relate it to the theorem's conclusion for solving \(x=Bx+c\).
Solution
\(g\) is a contraction in the \(\ell^2\)-induced norm iff \(\|B\|_2 \lt 1\), where \(\|B\|_2\) is the largest singular value of \(B\) (equivalently \(\sqrt{\lambda_{\max}(B^{\mathsf T}B)}\)); this is because \(d(g(x),g(y))=\|B(x-y)\|_2 \le \|B\|_2\|x-y\|_2\) with equality attained for some direction, so \(\|B\|_2\) is exactly the smallest valid Lipschitz constant. Since \((\mathbb{R}^2,\|\cdot\|_2)\) is complete and \(g\) is affine hence self-mapping on all of \(\mathbb{R}^2\), the theorem then guarantees: a unique \(x^\ast\) with \(x^\ast=Bx^\ast+c\) (equivalently \((I-B)x^\ast=c\) has a unique solution, consistent with \(1\) not being an eigenvalue of \(B\) since \(\|B\|_2\lt1\) bounds all eigenvalues' moduli below 1), and that \(x_n=B^nx_0 + (\text{terms in }c) \to x^\ast\) from every starting point, geometrically with ratio \(\|B\|_2\). - Construct an explicit example of a map \(g\) on a complete space \(X\) that has a unique fixed point which globally attracts every orbit, yet \(g\) is not a contraction with respect to any metric giving the same convergent behaviour on all of \(X\) — i.e. verify the converse genuinely fails, and explain what a numerical analyst should do given only convergent-looking numerics with no such certificate.
Solution
Take \(X=[0,\infty)\) (complete, closed subset of \(\mathbb{R}\)) and \(g(x) = \frac{x}{1+x}\). Then \(g(0)=0\) is the unique fixed point (solve \(x=x/(1+x) \Rightarrow x(1+x)=x \Rightarrow x^2=0 \Rightarrow x=0\)), and for any \(x_0\gt0\), \(x_{n+1}=x_n/(1+x_n) \lt x_n\), a decreasing sequence bounded below by \(0\), hence convergent, and one checks the limit \(\ell\) satisfies \(\ell=\ell/(1+\ell)\Rightarrow\ell=0\); so every orbit converges to \(0\). But \(g'(x) = 1/(1+x)^2\), which \(\to1\) as \(x\to0^+\), so \(\sup_{x\in X}|g'(x)|=1\) (approached, not exceeded, but not bounded below 1 either), meaning no \(L\lt1\) works uniformly on all of \(X\) — indeed \(d(g(x),g(0))/d(x,0) = 1/(1+x) \to 1\) as \(x\to0\), so the ratio gets arbitrarily close to \(1\) near the fixed point itself, the theorem's hypothesis genuinely fails. Given only numerical evidence that an iteration "looks convergent," an analyst cannot conclude a contraction certificate exists; the correct response is either to prove convergence by a different route (here, monotone-bounded-sequence argument, or Lyapunov-type methods) or restrict to a subdomain bounded away from the point where the ratio approaches 1 (e.g. \(X'=[\delta,\infty)\) for fixed \(\delta\gt0\), on which \(\sup|g'|=1/(1+\delta)^2\lt1\) does give a valid local contraction certificate).