The method of Lagrange multipliers

Theorem

The method of Lagrange multipliers

T-053Home MU-203Threads space · change

Statement

Let \( U \subseteq \mathbb{R}^n \) be open, let \( f : U \to \mathbb{R} \) and \( g = (g_1,\dots,g_m) : U \to \mathbb{R}^m \) (with \( m \lt n \)) be \( C^1 \) functions, and let \( M = \{ x \in U : g(x) = 0 \} \) be the constraint set. Suppose \( x_0 \in M \) is a local extremum (maximum or minimum) of \( f \) restricted to \( M \), and suppose the Jacobian \( Dg(x_0) \) has full rank \( m \) (equivalently \( \nabla g_1(x_0), \dots, \nabla g_m(x_0) \) are linearly independent). Then there exist scalars \( \lambda_1, \dots, \lambda_m \in \mathbb{R} \), the Lagrange multipliers, such that \[ \nabla f(x_0) = \sum_{i=1}^m \lambda_i \, \nabla g_i(x_0). \] Equivalently, \( x_0 \) is a critical point of the Lagrangian \( \mathcal{L}(x,\lambda) = f(x) - \sum_{i=1}^m \lambda_i g_i(x) \), i.e. \( \nabla_x \mathcal{L}(x_0,\lambda) = 0 \) and \( \nabla_\lambda \mathcal{L}(x_0,\lambda) = -g(x_0) = 0 \).

Why it matters

Most optimisation problems that arise in geometry, physics and economics are not free extremisations over all of \( \mathbb{R}^n \) but extremisations subject to side conditions — a particle confined to a surface, a consumer's budget line, a probability vector that must sum to one. The method converts a constrained problem in \( n \) variables into an unconstrained algebraic system in \( n+m \) variables (the original variables plus the multipliers), replacing geometric reasoning about tangency with the solvable equation \( \nabla f = \sum \lambda_i \nabla g_i \).

The theorem is also the prototype for a much larger idea in analysis: extremising a functional subject to a constraint forces the "unconstrained gradient" to lie in the space orthogonal to the constraint's tangent directions. This same principle reappears, in infinite dimensions, in the calculus of variations (isoperimetric problems), in the Karush–Kuhn–Tucker conditions of convex optimisation, and in the Euler–Lagrange equations of classical mechanics with constraints.

Hypotheses

\( f \) and \( g \) are \( C^1 \) on an open set \( U \). Without differentiability, \( \nabla f \) and \( \nabla g_i \) need not exist, so the equation \( \nabla f = \sum \lambda_i \nabla g_i \) is not even meaningful. E.g. \( f(x,y) = |x| \) on \( M = \{y=0\} \) has a minimum at the origin, but \( \nabla f(0,0) \) does not exist, and no linear tangency statement in terms of gradients is available there.

\( x_0 \) is a local extremum of \( f|_M \) (not merely any point of \( M \)). The conclusion asserts a relation that holds at extremisers; at a generic point of \( M \) there is no reason for \( \nabla f \) to lie in the span of the \( \nabla g_i \). E.g. on \( M = \{x^2+y^2=1\} \) with \( f(x,y) = x \), the point \( (0,1) \) is not an extremum of \( f|_M \), and indeed \( \nabla f(0,1) = (1,0) \) is not parallel to \( \nabla g(0,1) = (0,2) \).

Constraint qualification: \( Dg(x_0) \) has full rank \( m \) (the gradients \( \nabla g_1(x_0),\dots,\nabla g_m(x_0) \) are linearly independent). This is the regularity hypothesis that makes \( M \) a genuine \( (n-m) \)-dimensional manifold near \( x_0 \) (by the implicit function theorem) and makes the multipliers exist and be unique. Drop it: take \( g(x,y) = x^3 - y^2 \) so \( M \) is the cusp curve \( y^2 = x^3 \), and \( f(x,y) = x \). The origin minimises \( f \) on \( M \) (since \( x \geq 0 \) on \( M \)), but \( \nabla g(0,0) = (0,0) \), so no \( \lambda \) can satisfy \( \nabla f(0,0) = \lambda \nabla g(0,0) = (1,0) = \lambda(0,0) \). The theorem's conclusion simply fails at this singular point.

Proof

\text{By the constraint qualification, } Dg(x_0) \text{ has rank } m.

Hypothesis; after relabelling coordinates \( x = (u,v) \) with \( u \in \mathbb{R}^{n-m} \), \( v \in \mathbb{R}^m \), the \( m \times m \) block \( \partial g / \partial v (x_0) \) is invertible. A

\exists \text{ open } V \ni u_0,\ W \ni v_0,\ \text{and } C^1\ h : V \to W \text{ with } g(u,h(u)) = 0 \text{ on } V,\ h(u_0)=v_0.

This is exactly the hypothesis of the Implicit Function Theorem (invertible partial Jacobian at \( x_0 \)); it produces a local \( C^1 \) parametrisation of \( M \) near \( x_0 \) as the graph \( v = h(u) \). B t3

\varphi(u) := (u, h(u)) \text{ satisfies } D\varphi(u_0) = \begin{pmatrix} I_{n-m} \\ Dh(u_0) \end{pmatrix},\quad g(\varphi(u)) \equiv 0.

Definition of \( \varphi \) as a local chart of \( M \); its columns \( \partial \varphi/\partial u_j (u_0) \) span the tangent space \( T_{x_0}M \), since \( \varphi \) is a \( C^1 \) embedding onto a neighbourhood of \( x_0 \) in \( M \) (rank \( n-m \), injective by construction). B

\Phi(u) := f(\varphi(u)) \text{ has an unconstrained local extremum at } u_0 \text{ in } \mathbb{R}^{n-m}.

\( \varphi \) is a local homeomorphism of \( V \) onto a neighbourhood of \( x_0 \) in \( M \), so "extremum of \( f \) on \( M \) near \( x_0 \)" transports exactly to "extremum of \( \Phi = f\circ\varphi \) on \( V \) near \( u_0 \)". A

D\Phi(u_0) = 0,\ \text{i.e.}\ \nabla f(x_0)^{\!\top} D\varphi(u_0) = 0.

\( \Phi \) is \( C^1 \) (composite of \( C^1 \) maps) and has an interior unconstrained local extremum at \( u_0 \), so by Fermat's theorem (first-derivative test for extrema of differentiable functions on open sets) \( D\Phi(u_0) = 0 \); the chain rule gives \( D\Phi(u_0) = Df(x_0)\,D\varphi(u_0) \). B

\nabla f(x_0) \perp T_{x_0}M, \quad \text{where } T_{x_0}M = \operatorname{Im} D\varphi(u_0) = \operatorname{Ker} Dg(x_0).

Step 5 says \( \nabla f(x_0) \) annihilates every column of \( D\varphi(u_0) \), i.e. every tangent vector to \( M \) at \( x_0 \); and differentiating \( g(\varphi(u))\equiv 0 \) at \( u_0 \) gives \( Dg(x_0) D\varphi(u_0) = 0 \), so \( \operatorname{Im} D\varphi(u_0) \subseteq \operatorname{Ker} Dg(x_0) \), and a dimension count (\( \operatorname{rank} D\varphi(u_0) = n-m \) by step 3, \( \dim \operatorname{Ker} Dg(x_0) = n - \operatorname{rank} Dg(x_0) = n-m \) by step 1) forces equality. C t3

\operatorname{Ker} Dg(x_0) = \left(\operatorname{span}\{\nabla g_1(x_0),\dots,\nabla g_m(x_0)\}\right)^{\perp}.

Immediate from the definition of the Jacobian: \( Dg(x_0) w = 0 \iff \nabla g_i(x_0)\cdot w = 0 \) for every \( i=1,\dots,m \), i.e. \( w \) is orthogonal to each row gradient. A

\nabla f(x_0) \in \left(\operatorname{Ker} Dg(x_0)\right)^{\perp} = \operatorname{span}\{\nabla g_1(x_0),\dots,\nabla g_m(x_0)\}.

Steps 6–7 give \( \nabla f(x_0) \) orthogonal to \( \operatorname{Ker} Dg(x_0) \); taking orthogonal complements and using the standard linear-algebra fact \( (S^\perp)^\perp = \operatorname{span}(S) \) for a finite set \( S \) in \( \mathbb{R}^n \) (with \( \nabla g_i(x_0) \) linearly independent by hypothesis, so \( \operatorname{span}\{\nabla g_i(x_0)\} \) already equals its own double perp) identifies the double complement with the span of the constraint gradients. C

\therefore\ \exists\, \lambda_1,\dots,\lambda_m \in \mathbb{R} : \ \nabla f(x_0) = \sum_{i=1}^m \lambda_i \nabla g_i(x_0).

Unpacking membership of the span in step 8 gives exactly the multiplier equation; since \( \{\nabla g_i(x_0)\} \) is linearly independent (hypothesis), the coefficients \( \lambda_i \) are moreover unique. A

Result

\nabla f(x_0) = \sum_{i=1}^{m} \lambda_i\, \nabla g_i(x_0) \quad \text{at any regular constrained extremum } x_0

Reading. At a point where \( f \) is extremised subject to \( g=0 \), the gradient of \( f \) has no component tangent to the constraint surface — if it did, moving along that tangent direction (which stays on the constraint set to first order) would increase or decrease \( f \), contradicting extremality. So \( \nabla f \) must be a linear combination of the constraint gradients, which together span the space normal to the surface.

Scope. Applies to \( C^1 \) objective and constraints on an open subset of \( \mathbb{R}^n \), at points where the constraint gradients are linearly independent (the constraint set is a smooth manifold there). It gives a necessary first-order condition only — it locates candidates, and a second-order test (bordered Hessian) or direct comparison is needed to classify them as maxima, minima, or saddle-type critical points of the constrained problem.

Corollaries & converses

Single constraint (m=1): \( \nabla f(x_0) = \lambda \nabla g(x_0) \) — the classical statement that the level curves of \( f \) and \( g \) are tangent at the extremum.
Converse fails: a point satisfying \( \nabla f(x_0) = \sum \lambda_i \nabla g_i(x_0) \) (a Lagrange critical point) need not be an extremum — it may be a constrained saddle. E.g. \( f(x,y)=x^2-y^2 \) on \( M=\{x^2+y^2=1\} \): the multiplier equation holds at all four points \( (\pm1,0), (0,\pm1) \), but \( (\pm1,0) \) are maxima and \( (0,\pm1) \) are minima of \( f|_M \) — all are critical, not all extremal.
Existence is not guaranteed globally: the theorem is local and conditional on an extremum existing; on a non-compact \( M \), \( f|_M \) may have no extremum at all even though Lagrange points exist (or vice versa).
Combined with the Extreme Value Theorem (continuous \( f \) on compact \( M \) attains a max and min), the theorem gives a complete algorithm: solve the Lagrange system, evaluate \( f \) at every regular solution (and separately at any singular points of \( M \)), and take the largest/smallest value.

Fails without

Constraint qualification dropped: \( g(x,y) = x^3-y^2 \), \( f(x,y)=x \), extremum at the cuspidal origin — \( \nabla g(0,0)=(0,0) \) so \( \nabla f(0,0)=(1,0) \) cannot be written as \( \lambda\cdot(0,0) \). No multiplier exists even though the origin genuinely minimises \( f \) on \( M \).
\( C^1 \) hypothesis dropped: \( f(x,y)=-|x| \) constrained to \( g(x,y)=y=0 \); the maximum of \( f|_M \) is at \( (0,0) \), but \( f \) is not differentiable there, so \( \nabla f(0,0) \) does not exist and the multiplier equation cannot even be stated.
Extremum hypothesis dropped (arbitrary critical point mistaken for extremum): as in the corollaries, \( f(x,y)=x^2-y^2 \) on the unit circle has Lagrange points at \( (0,\pm1) \) that are saddle points of the unconstrained problem yet minima of the constrained one, and the method alone (without classification) cannot distinguish them from maxima.

Common errors

Treating every solution of the Lagrange system as automatically a maximum (or automatically a minimum) without checking — the method only produces candidates; classification requires comparing values or a second-order test.
Forgetting to check the rank/constraint-qualification hypothesis, then being surprised when a genuine extremum (typically at a singular or cusp point of \( M \)) is missed by the multiplier equations.
Sign errors in setting up \( \mathcal{L} = f - \sum \lambda_i g_i \) versus \( f + \sum \lambda_i g_i \): harmless for locating critical points (just flips the sign of \( \lambda \)) but a frequent source of confusion when interpreting \( \lambda \) as a shadow price/sensitivity.
On a bounded region with both an equality constraint and a boundary, forgetting to separately check the boundary of \( M \) itself (where \( M \) is not open in the ambient constraint manifold) — Lagrange's theorem is an interior-type first-order condition and says nothing at boundary/corner points of \( M \).
Applying the method to inequality constraints \( g(x)\le 0 \) as if they were equalities, ignoring that an inequality-constrained extremum may occur strictly inside the feasible region, where no multiplier condition applies at all (that regime needs the Karush–Kuhn–Tucker conditions).

Discussion

The method is due to Joseph-Louis Lagrange, introduced in his 1788 Mécanique analytique as a way to handle constrained mechanical systems without eliminating variables by substitution. Its deep insight is geometric rather than algebraic: constrained extremisation is fundamentally about tangency, and the gradient condition is simply the coordinate-free statement that the unconstrained "uphill direction" has been squeezed entirely into the directions normal to the constraint.

The proof given here follows the "reduce to unconstrained calculus via a local chart" strategy, which is honest but conceals a choice: which \( m \) of the \( n \) coordinates to solve for as functions of the rest. The full-rank hypothesis on \( Dg(x_0) \) is precisely what guarantees at least one such choice works, via the implicit function theorem — the constraint qualification is not a technicality bolted on for rigour, it is the load-bearing hypothesis that makes the proof's central step (parametrising \( M \) as a graph) possible at all.

The multipliers \( \lambda_i \) are not bookkeeping devices — they have a quantitative meaning under mild extra hypotheses (the envelope theorem): if the constraint is perturbed to \( g_i(x) = c_i \) and \( f^*(c) \) denotes the extremal value as a function of \( c \), then \( \partial f^*/\partial c_i = -\lambda_i \) at \( c=0 \), so each multiplier measures the marginal sensitivity of the optimum to relaxing that constraint — the origin of the "shadow price" interpretation used throughout economics.

A common misconception is that the theorem asserts \( \nabla f \) and \( \nabla g \) point in the "same direction"; in fact only linear dependence (parallelism up to sign and scale, or membership in a span for \( m\gt1 \)) is asserted, and \( \lambda_i \) can be any real number including zero or negative. A second misconception is conflating this local, first-order, necessary condition with sufficiency for a global extremum; global conclusions require comparing all Lagrange points (plus non-regular points of \( M \)) using an independent existence result such as the Extreme Value Theorem on a compact \( M \).

Worked examples

\text{Maximise } f(x,y) = xy \text{ subject to } g(x,y) = x+y-10 = 0.

Set up the constrained optimisation; \( M = \{x+y=10\} \) is a line, so \( \nabla g = (1,1) \neq 0 \) everywhere — the constraint qualification holds globally. A

\nabla f = (y,x), \qquad \nabla f = \lambda \nabla g \implies y = \lambda,\ x = \lambda.

Apply the theorem's conclusion directly (single constraint, so a single scalar \( \lambda \)). A

x = y \quad\text{and}\quad x+y=10 \implies x=y=5.

Solve the Lagrange system \( \{x=\lambda, y=\lambda, x+y=10\} \) simultaneously — this is pure algebra, not part of the theorem itself. A

\text{As } x \to \pm\infty \text{ along } M,\ f(x,10-x) = 10x - x^2 \to -\infty, \text{ so } (5,5) \text{ is the global maximum.}

The theorem only certifies \( (5,5) \) as a critical point of \( f|_M \); classification uses that \( f|_M \) is a downward parabola in the parameter \( x \), which \to -\infty at both ends, so its unique critical point is a global max. B

\max_{x+y=10} xy = 25, \text{ attained at } (x,y)=(5,5)

\text{Extremise } f(x,y,z) = x + y + z \text{ on the sphere } g(x,y,z)=x^2+y^2+z^2-1=0.

\( M \) is the unit sphere in \( \mathbb{R}^3 \), compact, and \( \nabla g = (2x,2y,2z) \neq 0 \) for all \( (x,y,z)\in M \) (since \( 0 \notin M \)) — constraint qualification holds everywhere, and the Extreme Value Theorem guarantees global extrema exist. A

\nabla f = (1,1,1) = \lambda(2x,2y,2z) \implies x=y=z=\tfrac{1}{2\lambda}.

Apply the theorem's conclusion for \( m=1 \). A

x=y=z \ \text{and}\ 3x^2 = 1 \implies x = \pm \tfrac{1}{\sqrt{3}}, \quad (x,y,z) = \pm\left(\tfrac{1}{\sqrt3},\tfrac{1}{\sqrt3},\tfrac{1}{\sqrt3}\right).

Substitute \( x=y=z \) into the constraint \( x^2+y^2+z^2=1 \) and solve. A

f\left(\tfrac{1}{\sqrt3},\tfrac{1}{\sqrt3},\tfrac{1}{\sqrt3}\right) = \sqrt3, \qquad f\left(-\tfrac{1}{\sqrt3},-\tfrac{1}{\sqrt3},-\tfrac{1}{\sqrt3}\right) = -\sqrt3.

Since \( M \) is compact and \( f \) continuous, the max and min exist (Extreme Value Theorem) and, by step 1, must occur at the only two Lagrange candidates — so these are the global extreme values, no further check needed. B

\max_{x^2+y^2+z^2=1}(x+y+z) = \sqrt3, \qquad \min_{x^2+y^2+z^2=1}(x+y+z) = -\sqrt3

Problems

Find the extreme values of \( f(x,y) = x^2+y^2 \) subject to \( g(x,y) = xy - 1 = 0 \) (with \( x\gt0 \)), and classify them.
Solution
\( \nabla f = (2x,2y) = \lambda(y,x) \). From \( 2x=\lambda y \) and \( 2y = \lambda x \), multiply: \( 4xy = \lambda^2 xy \), and since \( xy=1\neq0 \), \( \lambda^2=4 \), \( \lambda=\pm2 \). If \( \lambda=2 \): \( 2x=2y \Rightarrow x=y \), with \( xy=1,x\gt0 \) gives \( x=y=1 \), \( f=2 \). If \( \lambda=-2 \): \( x=-y \), impossible with \( x\gt0,y=1/x\gt0 \). So the only candidate is \( (1,1) \) with \( f=2 \). Along \( M \) (parametrise \( y=1/x \), \( x\gt0 \)), \( f(x)=x^2+1/x^2 \to \infty \) as \( x\to0^+ \) or \( x\to\infty \), and \( f \) is smooth with a unique interior critical point, so \( (1,1) \) is a global minimum, \( f_{\min}=2 \); there is no maximum (unbounded above).
Explain, using the theorem's hypotheses, why the method of Lagrange multipliers fails to find the minimum of \( f(x,y)=x^2+y^2 \) on \( M = \{(x,y): x^3=y^2, x\ge0\} \) at the origin.
Solution
Write \( g(x,y)=x^3-y^2 \). At \( (0,0) \), \( \nabla g(0,0) = (3x^2,-2y)|_{(0,0)} = (0,0) \), so the constraint qualification (full rank of \( Dg \)) fails there — \( M \) has a cusp at the origin and is not a smooth 1-manifold there. The origin is clearly the global minimum of \( f \) on \( M \) (\( f\ge0=f(0,0) \)), but the equation \( \nabla f(0,0) = \lambda \nabla g(0,0) \) reads \( (0,0) = \lambda(0,0) \), which holds for every \( \lambda \) and gives no information — the theorem's proof breaks down at step 1 (no invertible \( 1\times1 \) block of \( Dg \) exists at the origin), so it correctly does not — and cannot — certify this point.
Use Lagrange multipliers to find the point(s) on the plane \( x+2y+3z=6 \) closest to the origin, and state which theorem guarantees the minimum exists.
Solution
Minimise \( f=x^2+y^2+z^2 \) (squared distance) subject to \( g=x+2y+3z-6=0 \). \( \nabla f=(2x,2y,2z)=\lambda(1,2,3) \Rightarrow x=\lambda/2,\ y=\lambda,\ z=3\lambda/2 \). Substitute: \( \lambda/2 + 2\lambda + 9\lambda/2 = 6 \Rightarrow (1/2+2+9/2)\lambda=6 \Rightarrow 7\lambda=6 \Rightarrow \lambda=6/7 \). So \( (x,y,z)=(3/7, 6/7, 9/7) \). Existence: the plane is not compact, so the Extreme Value Theorem doesn't directly apply; instead note \( f\to\infty \) as \( \|(x,y,z)\|\to\infty \) along the plane (coercivity), and \( f \) is continuous, so a global minimum exists on the (closed) plane, and by uniqueness of the Lagrange solution it must occur at \( (3/7,6/7,9/7) \), giving minimum distance \( \sqrt{9/49+36/49+81/49}=\sqrt{126/49}=\tfrac{3\sqrt{14}}{7} \).
Two constraints: extremise \( f(x,y,z)=x+y+z \) subject to \( g_1(x,y,z)=x^2+y^2-2=0 \) and \( g_2(x,y,z)=x+z-1=0 \). Set up and solve the Lagrange system, checking the constraint qualification.
Solution
\( \nabla g_1=(2x,2y,0) \), \( \nabla g_2=(1,0,1) \); these are linearly independent except where \( y=0 \) and simultaneously the two vectors are parallel — direct check shows they are independent whenever \( (x,y)\ne(0,0) \), which holds throughout \( M \) since \( x^2+y^2=2 \). The Lagrange equation \( \nabla f=\lambda_1\nabla g_1+\lambda_2\nabla g_2 \) gives \( 1=2\lambda_1 x+\lambda_2 \), \( 1=2\lambda_1 y \), \( 1=\lambda_2 \). From the third, \( \lambda_2=1 \); then the first gives \( 2\lambda_1 x=0 \). Case \( \lambda_1=0 \): second equation gives \( 1=0 \), contradiction. So \( x=0 \), and \( g_1 \) gives \( y^2=2\Rightarrow y=\pm\sqrt2 \); \( g_2 \) gives \( z=1 \). Candidates: \( (0,\sqrt2,1) \) with \( f=1+\sqrt2 \), and \( (0,-\sqrt2,1) \) with \( f=1-\sqrt2 \). Since \( M \) (intersection of a cylinder and a plane) is a compact curve (an ellipse) and \( f \) continuous, the Extreme Value Theorem guarantees global extrema exist among these two candidates: max \( 1+\sqrt2 \), min \( 1-\sqrt2 \).
A student claims: "At every point where \( \nabla f = \lambda \nabla g \), \( f \) restricted to \( g=0 \) has either a local max or local min." Give a counterexample and identify exactly which step of the theorem's proof this claim misunderstands.
Solution
Counterexample: \( f(x,y)=x^3 \), \( g(x,y)=y=0 \). Then \( \nabla f=(3x^2,0) \), \( \nabla g=(0,1) \), and the Lagrange equation \( (3x^2,0)=\lambda(0,1) \) forces \( 3x^2=0 \) and \( \lambda=0 \), so \( x=0 \) is the unique candidate. But on \( M=\{y=0\} \), \( f(x,0)=x^3 \) is strictly increasing through \( x=0 \) — an inflection, not a local extremum in either direction. The claim misreads the theorem as an "if and only if": the theorem proves only the forward implication (extremum \( \Rightarrow \) Lagrange condition, via Fermat's theorem applied to \( \Phi=f\circ\varphi \) in step 5), not the converse. \( \Phi(u)=u^3 \) indeed has \( \Phi'(0)=0 \) without \( u=0 \) being an extremum of \( \Phi \) — exactly analogous to how a vanishing derivative in single-variable calculus flags a critical point, not automatically an extremum.