Orthogonality and Gram–Schmidt

The Geometric Intuition

Imagine you have a set of vectors that span a space, but they are "tilted" relative to each other. If you want to find how much of a new vector lies in the direction of the first basis vector, the tilt complicates the calculation because the projection onto the first vector also has a component along the second. Orthogonality removes this coupling. When vectors are orthogonal, the projection of a vector onto one basis element is completely independent of the others. Gram–Schmidt is a sequential "straightening" process. We take the first vector and normalize it. Then we take the second vector, subtract its shadow (projection) onto the first, and normalize the remainder. This remainder is now perpendicular to the first. We repeat this for the third vector, subtracting its shadows onto both previous orthogonal vectors. The result is a set of vectors that are mutually perpendicular and of unit length, yet they span exactly the same subspace as the original set. No information is lost; only the coordinate axes are rotated and scaled to be convenient.

The Projection Formula

The core operation in Gram–Schmidt is the projection of a vector \(v\) onto a subspace spanned by an orthonormal set \(\{e_1, \dots, e_{k-1}\}\). Because the basis is orthonormal, the projection is simply the sum of the projections onto each individual basis vector. For a single orthonormal vector \(e_j\), the projection of \(v\) onto \(e_j\) is given by \(\langle e_j, v \rangle e_j\). Note the order of the inner product: in the complex case, the scalar is on the left if we follow the physics convention where the inner product is linear in the second argument, or we must be careful with conjugation. Assuming the standard physics convention \(\langle u, v \rangle\) is linear in \(v\) and conjugate-linear in \(u\), the projection of \(v\) onto \(e_j\) is \(\langle e_j, v \rangle e_j\).

However, during the Gram–Schmidt process, we are constructing the orthonormal set. At step \(k\), we have already constructed \(e_1, \dots, e_{k-1}\). We have the original vector \(v_k\). We want to remove the components of \(v_k\) that lie in the span of the previous vectors. The component of \(v_k\) in the direction of \(e_j\) is \(\langle e_j, v_k \rangle e_j\). Therefore, the orthogonal part \(u_k\) is:

Once we have this orthogonal vector \(u_k\), we normalize it to get \(e_k = u_k / \lVert u_k \rVert\). This ensures \(\langle e_k, e_k \rangle = 1\).

The Gram–Schmidt Algorithm

Let \(\{v_1, \dots, v_n\}\) be a linearly independent set. We construct \(\{e_1, \dots, e_n\}\) as follows:

1. Set \(u_1 = v_1\). Define \(e_1 = u_1 / \lVert u_1 \rVert\).

2. For \(k = 2\) to \(n\):

a. Compute \(u_k = v_k - \sum_{j=1}^{k-1} \langle e_j, v_k \rangle e_j\).

b. If \(\lVert u_k \rVert = 0\), the vectors are linearly dependent (contradiction to premise), so we stop. Otherwise, define \(e_k = u_k / \lVert u_k \rVert\).

This process guarantees that \(\langle e_i, e_j \rangle = \delta_{ij}\). The key property is that \(\operatorname{span}\{e_1, \dots, e_k\} = \operatorname{span}\{v_1, \dots, v_k\}\). This means we have not changed the subspace we are working in, only the basis. In quantum mechanics, this is crucial when we want to express a state in a basis of eigenstates of a Hermitian operator. If the eigenvectors are not initially orthogonal (which they must be for distinct eigenvalues of a Hermitian operator, but numerical errors or degenerate subspaces may require explicit orthogonalization), Gram–Schmidt provides the rigorous method to enforce this orthogonality.

Connection to Self-Adjoint Operators

The spectral theorem states that for a self-adjoint operator \(A\) on a finite-dimensional inner product space, there exists an orthonormal basis of eigenvectors. The Gram–Schmidt process is the constructive tool that allows us to find such a basis within degenerate eigenspaces. If \(A v = \lambda v\) and \(A w = \lambda w\) for the same \(\lambda\), then \(v\) and \(w\) are not necessarily orthogonal. However, any linear combination \(a v + b w\) is also an eigenvector with eigenvalue \(\lambda\). We can apply Gram–Schmidt to the basis of the eigenspace to produce an orthonormal basis of eigenvectors. This is why the spectral theorem is so powerful: it guarantees that the "geometry" of the operator is compatible with the "geometry" of the space. The operator does not twist the space in a way that breaks orthogonality; it preserves the angles between eigenvectors of distinct eigenvalues and allows us to orthogonalize those of the same eigenvalue.