The Min-Max Principle and Eigenvalue Bounds

The Core Idea

Imagine you are trying to find the lowest point in a landscape, but you are not allowed to look at the whole map. You can only walk on a specific path. The lowest point you find on that path is an upper bound for the true lowest point of the entire landscape. If you restrict your search to a larger area, your best guess can only get better (lower). Now, imagine you want to find the second lowest point. You must ignore the absolute lowest point. The min-max principle formalizes this intuition: to find the \(n\)-th eigenvalue, you look at all possible \(n\)-dimensional subspaces, and within each, you find the maximum possible Rayleigh quotient. The minimum of these maxima gives you the exact \(n\)-th eigenvalue. It is a game of "best worst-case scenario" that yields a precise mathematical result.

Formal Statement of the Principle

Let \(\hat{A}\) be a self-adjoint operator on a Hilbert space \(\mathcal{H}\) with a discrete, bounded-below spectrum. Let the eigenvalues be ordered such that \(\lambda_1 \le \lambda_2 \le \lambda_3 \le \cdots\). The Rayleigh quotient for a non-zero vector \(\psi\) is defined as \(R(\psi) = \dfrac{\langle \psi, \hat{A}\psi\rangle}{\langle \psi,\psi\rangle}\). The Courant–Fischer min–max principle states that the \(n\)-th eigenvalue can be characterized in two equivalent ways: as the minimum over all \(n\)-dimensional subspaces of the maximum Rayleigh quotient within that subspace, or as the maximum over all \((n-1)\)-dimensional subspaces of the minimum Rayleigh quotient for vectors orthogonal to that subspace.

This duality is crucial. The first expression tells us that if we pick any \(n\)-dimensional subspace \(S\), the largest eigenvalue of the projection of \(\hat{A}\) onto \(S\) is at least \(\lambda_n\). The second expression tells us that if we remove any \((n-1)\)-dimensional subspace \(T\), the smallest eigenvalue of \(\hat{A}\) in the remaining orthogonal complement is at most \(\lambda_n\).

Deriving Bounds for Approximations

In practical applications, we often approximate the Hilbert space with a finite-dimensional subspace \(V\) of dimension \(N\). If we compute the eigenvalues \(\mu_1 \le \mu_2 \le \cdots \le \mu_N\) of the operator restricted to \(V\), the min-max principle guarantees that \(\mu_k \ge \lambda_k\) for all \(k \le N\). This is the basis of the Rayleigh-Ritz method. The approximation provides an upper bound for the true eigenvalues. To improve the bound for \(\lambda_1\), we simply need to choose a subspace \(V\) that contains a vector close to the true ground state eigenvector. The quality of the bound depends entirely on how well the subspace captures the relevant physics.

For the \(n\)-th eigenvalue, the condition is stricter. We need a subspace of dimension \(n\) that is "aligned" with the first \(n\) eigenvectors. If the trial subspace is exactly the span of the first \(n\) true eigenvectors, the maximum Rayleigh quotient in it is exactly \(\lambda_n\). Any other \(n\)-dimensional subspace gives a maximum greater than or equal to \(\lambda_n\), still a valid upper bound.

Connection to Variational Methods

The min-max principle generalizes the simple variational principle for the ground state, which states that \(\lambda_1 = \min_{\psi \neq 0} R(\psi)\). While the ground state energy is easy to bound from above, excited states are notoriously difficult because the variational principle alone does not distinguish between the ground state and excited states. The min-max principle resolves this by introducing the dimension of the subspace as a parameter. It ensures that we are not accidentally finding the ground state again when looking for the first excited state. This is why in quantum chemistry, finding the first excited state often requires imposing orthogonality constraints to the ground state wavefunction, effectively applying the second part of the min-max formula.