Spectral Theorem for Hermitian Observables

Derivation

Spectral Theorem for Hermitian Observables

D-213 Home PU-301 Threads chance · symmetry Depends on hilbert-space-inner-product-axioms, born-rule-probability

Statement

Let \(\hat{A}\) be a bounded self-adjoint (Hermitian) operator on a finite-dimensional complex Hilbert space \(\mathcal{H}\), meaning \(\hat{A}=\hat{A}^{\dagger}\) so that \(\langle \phi\,|\,\hat{A}\psi\rangle=\langle \hat{A}\phi\,|\,\psi\rangle\) for all \(\psi,\phi\in\mathcal{H}\). Then every eigenvalue of \(\hat{A}\) is real, eigenvectors belonging to distinct eigenvalues are orthogonal, and \(\mathcal{H}\) admits an orthonormal basis \(\{|a_n\rangle\}\) composed entirely of eigenvectors of \(\hat{A}\). Equivalently \(\hat{A}=\sum_n a_n\,\hat{P}_n\) with real \(a_n\) and orthogonal projectors \(\hat{P}_n=\hat{P}_n^{\dagger}=\hat{P}_n^{2}\) satisfying \(\sum_n\hat{P}_n=\hat{\mathbb{1}}\).

Why it matters

The measurement postulate of quantum mechanics asserts that the possible outcomes of measuring an observable are the eigenvalues of a Hermitian operator, and that after measurement the state collapses onto the corresponding eigenspace. The spectral theorem is what makes this postulate coherent: it guarantees the outcomes are real numbers (as any physical reading must be) and that the eigenprojectors resolve the identity, so the Born-rule probabilities over all outcomes sum to one.

It also supplies the mathematical scaffolding for the entire formalism: functional calculus \(f(\hat{A})=\sum_n f(a_n)\hat{P}_n\), the definition of expectation values, the diagonalisation that turns dynamics into phases \(e^{-iEt/\hbar}\), and the compatibility of commuting observables through a shared eigenbasis. Without it, "expand the state in eigenstates" would be an unjustified article of faith.

Assumptions

\(\hat{A}\) is self-adjoint, \(\hat{A}=\hat{A}^{\dagger}\), not merely symmetric.If only symmetric (Hermitian on a dense domain but with unequal deficiency indices), real eigenvalues survive but completeness can fail; no eigenbasis need exist and the measurement postulate loses its spectral support.

The inner product is positive-definite and sesquilinear (from hilbert-space-inner-product-axioms).Orthogonality and the norm \(\lVert\psi\rVert^{2}=\langle\psi|\psi\rangle\ge 0\) would be meaningless; the reality proof and Gram–Schmidt both collapse without \(\langle\psi|\psi\rangle>0\) for \(\psi\ne 0\).

\(\mathcal{H}\) is finite-dimensional (or \(\hat{A}\) is compact self-adjoint).In infinite dimensions a general self-adjoint operator may have continuous spectrum with no normalisable eigenvectors; the sum becomes a projection-valued integral \(\hat{A}=\int a\,d\hat{P}(a)\).

The characteristic polynomial factorises fully over \(\mathbb{C}\) (fundamental theorem of algebra).The existence of at least one eigenvalue is not automatic over \(\mathbb{R}\); complex closure is what seeds the induction that builds the basis.

Derivation

\[ \hat{A}\,|a\rangle = a\,|a\rangle,\qquad |a\rangle\neq 0 \]

Over \(\mathbb{C}\) the characteristic polynomial \(\det(\hat{A}-a\hat{\mathbb{1}})=0\) has a root, so at least one eigenpair exists. A

\[ \langle a\,|\,\hat{A}\,|a\rangle = a\,\langle a|a\rangle \]

Left-multiply the eigenvalue equation by the bra \(\langle a|\); linearity of the inner product in the ket. A

\[ \overline{\langle a\,|\,\hat{A}\,|a\rangle} = \langle a\,|\,\hat{A}^{\dagger}\,|a\rangle = \langle a\,|\,\hat{A}\,|a\rangle \]

Conjugate-symmetry of the inner product defines the adjoint; self-adjointness \(\hat{A}^{\dagger}=\hat{A}\) then makes the expectation value equal to its own complex conjugate. B

\[ a\,\langle a|a\rangle = \overline{a}\,\langle a|a\rangle \;\Longrightarrow\; (a-\overline{a})\,\langle a|a\rangle = 0 \]

Substitute step 2 into step 3 and use \(\overline{\langle a|a\rangle}=\langle a|a\rangle\) (the norm is real). A

\[ \langle a|a\rangle > 0 \;\Longrightarrow\; a=\overline{a}\;\Longrightarrow\; a\in\mathbb{R} \]

Positive-definiteness of the inner product forbids the trivial factor; hence the imaginary part vanishes. Eigenvalues are real. A

\[ \hat{A}|a\rangle=a|a\rangle,\quad \hat{A}|b\rangle=b|b\rangle,\quad a\neq b \]

Take two eigenpairs with distinct eigenvalues to test orthogonality. A

\[ \langle b\,|\,\hat{A}\,|a\rangle = a\,\langle b|a\rangle, \qquad \langle b\,|\,\hat{A}\,|a\rangle = \langle \hat{A}b\,|\,a\rangle = b\,\langle b|a\rangle \]

Evaluate the same matrix element two ways: acting right, and moving \(\hat{A}\) left via self-adjointness with \(\overline{b}=b\) (step 5). B

\[ (a-b)\,\langle b|a\rangle = 0 \;\Longrightarrow\; \langle b|a\rangle = 0 \]

Subtract the two expressions; since \(a\neq b\), the overlap must vanish. Distinct-eigenvalue eigenvectors are orthogonal. A

\[ \mathcal{H}=\mathrm{span}\{|a\rangle\}\oplus |a\rangle^{\perp},\qquad \hat{A}\big(|a\rangle^{\perp}\big)\subseteq |a\rangle^{\perp} \]

Self-adjointness makes the orthogonal complement of an eigenvector invariant: if \(\langle a|\psi\rangle=0\) then \(\langle a|\hat{A}\psi\rangle=\langle \hat{A}a|\psi\rangle=a\langle a|\psi\rangle=0\). C

\[ \hat{A}\big|_{|a\rangle^{\perp}} \text{ is self-adjoint on a space of dimension } \dim\mathcal{H}-1 \]

The restriction inherits self-adjointness; induction on dimension repeats steps 1–9 inside the complement. C

\[ \{|a_n\rangle\}_{n=1}^{\dim\mathcal{H}},\qquad \langle a_m|a_n\rangle=\delta_{mn} \]

Collecting one normalised eigenvector at each induction step (Gram–Schmidt within any degenerate eigenspace) yields a complete orthonormal eigenbasis. C

\[ \hat{\mathbb{1}}=\sum_n |a_n\rangle\langle a_n|,\qquad \hat{A}=\hat{A}\,\hat{\mathbb{1}}=\sum_n a_n\,|a_n\rangle\langle a_n| \]

Completeness gives the resolution of the identity; act \(\hat{A}\) on it and use \(\hat{A}|a_n\rangle=a_n|a_n\rangle\). Grouping equal eigenvalues defines \(\hat{P}_n=\sum_{k\in n}|a_k\rangle\langle a_k|\). B

Result

\[ \hat{A}=\sum_n a_n\,\hat{P}_n,\qquad a_n\in\mathbb{R},\qquad \hat{P}_n\hat{P}_m=\delta_{nm}\hat{P}_n,\qquad \sum_n\hat{P}_n=\hat{\mathbb{1}} \]

Reading. A Hermitian observable is fully specified by a set of real eigenvalues (the measurable values) and mutually orthogonal projectors onto their eigenspaces. Any state expands as \(|\psi\rangle=\sum_n\hat{P}_n|\psi\rangle\), and the Born rule assigns outcome \(a_n\) the probability \(\langle\psi|\hat{P}_n|\psi\rangle=\lVert\hat{P}_n\psi\rVert^{2}\). Because the projectors resolve the identity, these probabilities sum to \(\langle\psi|\psi\rangle=1\).

Units check. The relation is operator-valued: \(\hat{P}_n\) is dimensionless (a projector, \(\hat{P}_n^2=\hat{P}_n\)), so \(\hat{A}\) carries exactly the units of its eigenvalues \(a_n\) — for energy \([a_n]=\mathrm{J}\), for spin-\(z\) \([a_n]=\mathrm{J\,s}\) (\(\hbar/2\)). Both sides of \(\hat{A}=\sum_n a_n\hat{P}_n\) therefore share the units of the observable, and \(\sum_n\hat{P}_n=\hat{\mathbb{1}}\) is dimensionless as required.

Limiting cases

Non-degenerate spectrum: every \(\hat{P}_n=|a_n\rangle\langle a_n|\) has rank one, and the eigenbasis is unique up to phases.
Degenerate eigenvalue: \(\hat{P}_n\) has rank \(g_n>1\); the eigenbasis within that block is free up to a \(U(g_n)\) rotation — a residual symmetry that a second commuting observable can fix.
\(\hat{A}=c\,\hat{\mathbb{1}}\): a single eigenvalue \(c\) with \(\hat{P}=\hat{\mathbb{1}}\); every vector is an eigenvector.
Projector itself (\(\hat{A}=\hat{P}\)): eigenvalues collapse to \(\{0,1\}\), recovering a yes/no measurement.
Real symmetric matrix (real \(\mathcal{H}\)): the theorem specialises to orthogonal diagonalisation, \(\hat{A}=O\,\mathrm{diag}(a_n)\,O^{\mathsf T}\).

Breaks when

Merely symmetric, not self-adjoint (infinite dimensions). An operator symmetric on a domain with unequal deficiency indices \((n_+\neq n_-)\) has no self-adjoint extension; the momentum operator on a half-line \([0,\infty)\) is the classic case. Real eigenvalues can still appear, but no complete eigenbasis exists and the spectral decomposition fails.
Continuous spectrum. For position \(\hat{x}\) or a free-particle Hamiltonian the "eigenvectors" are non-normalisable distributions (\(\langle x|x'\rangle=\delta(x-x')\)). The discrete sum must be replaced by the projection-valued Stieltjes integral \(\hat{A}=\int a\,d\hat{P}(a)\); \(\sum_n\hat{P}_n\to\int d\hat{P}=\hat{\mathbb{1}}\).
Non-normal operators. If \([\hat{A},\hat{A}^{\dagger}]\neq 0\) (e.g. the ladder operator \(\hat{a}\)) the operator is not diagonalisable by any unitary; eigenvectors need not be orthogonal or even complete, and defective Jordan blocks can appear.
Unbounded operators without self-adjoint domains. Careless formal manipulation of \(\hat{p}=-i\hbar\,d/dx\) on a finite interval without fixing boundary conditions yields a family of inequivalent spectra; the "theorem" gives different answers depending on the (physically meaningful) domain choice.

Failure modes

Confusing symmetric with self-adjoint. Assuming \(\langle\phi|\hat{A}\psi\rangle=\langle\hat{A}\phi|\psi\rangle\) on a formal domain guarantees a spectral decomposition. In finite dimensions it does; in infinite dimensions the domains must match — this is the single most common graduate-level error.
Treating degenerate eigenvectors as automatically orthogonal. Steps 6–8 only give orthogonality between distinct eigenvalues. Within a degenerate block you must run Gram–Schmidt yourself; two eigenvectors sharing \(a_n\) can be non-orthogonal.
Normalising the projector but not the eigenvectors. Writing \(\hat{P}_n=|a_n\rangle\langle a_n|\) with an unnormalised \(|a_n\rangle\) breaks \(\hat{P}_n^2=\hat{P}_n\) and inflates Born probabilities.
Assuming a complex eigenvalue "cancels" in a real observable. Students sometimes carry a complex \(a\) through and only take the real part of \(\langle\hat{A}\rangle\); the theorem forbids the imaginary part existing at all.
Using \(\hat{A}=\hat{A}^{\dagger}\) to conclude \(\hat{A}\) is positive. Hermiticity gives real eigenvalues, not non-negative ones; positivity requires \(\langle\psi|\hat{A}|\psi\rangle\ge 0\) separately.

Discussion

The spectral theorem is the bridge between the abstract axioms of a Hilbert space and the operational content of quantum mechanics. The measurement postulate could have been stated purely operationally — "measurements return real numbers with probabilities that sum to one" — but the theorem shows this is precisely what self-adjointness forces. Reality of outcomes (step 5) and normalisation of the total probability (the completeness \(\sum_n\hat{P}_n=\hat{\mathbb{1}}\)) are not two independent postulates; they are twin consequences of one algebraic condition, \(\hat{A}=\hat{A}^{\dagger}\), interpreted through a positive-definite inner product.

The theorem also encodes the thread of symmetry. Degeneracy — a single eigenvalue with a multi-dimensional eigenspace — is almost never accidental; it is the fingerprint of a symmetry group commuting with \(\hat{A}\). The \(U(g_n)\) freedom to rotate within a degenerate block is exactly the representation space of that symmetry, and the hydrogen atom's \(n^2\)-fold degeneracy is the visible shadow of its hidden \(SO(4)\) symmetry. Commuting observables share a simultaneous eigenbasis precisely because their projectors can be refined together, which is why a complete set of commuting observables (a CSCO) labels states uniquely.

Through the thread of chance, the eigenprojectors are what turn a deterministic operator into a probability measure. The map \(a_n\mapsto\langle\psi|\hat{P}_n|\psi\rangle\) is a genuine probability distribution on the spectrum — non-negative because \(\langle\psi|\hat{P}_n|\psi\rangle=\lVert\hat{P}_n\psi\rVert^2\ge 0\), and normalised because the projectors resolve the identity. Expectation and variance follow as moments, \(\langle\hat{A}\rangle=\sum_n a_n\langle\psi|\hat{P}_n|\psi\rangle\), grounding the statistical predictions the theory is tested by.

At the deepest level the finite-dimensional argument is a shadow of the general spectral theorem for unbounded self-adjoint operators, where the sum becomes a projection-valued measure \(d\hat{P}(a)\) and Stone's theorem promotes it to the one-parameter unitary group \(e^{-i\hat{H}t/\hbar}\) governing time evolution. Self-adjointness — not mere symmetry — is the condition guaranteeing that this evolution is unitary and probability-conserving; the deficiency-index analysis of von Neumann is precisely the machinery that decides whether a formal Hermitian expression corresponds to a bona fide observable at all.

Common misconceptions. "Hermitian" and "self-adjoint" coincide only in finite dimensions; in infinite dimensions they differ by domain conditions, and that difference decides whether an observable exists. A real eigenvalue does not require the operator to be positive. And degeneracy does not give orthogonality for free — orthogonality across distinct eigenvalues is automatic, but within a shared eigenvalue it must be constructed.

Worked examples

\[ \hat{S}_x=\frac{\hbar}{2}\begin{pmatrix}0&1\\[2pt]1&0\end{pmatrix} \]

Spin-\(\tfrac12\) along \(x\): a Hermitian \(2\times2\) matrix. Verify \(\hat{S}_x^{\dagger}=\hat{S}_x\) (real symmetric). A

\[ \det\!\left(\hat{S}_x-\lambda\hat{\mathbb{1}}\right)=\lambda^2-\left(\tfrac{\hbar}{2}\right)^2=0 \;\Longrightarrow\; \lambda=\pm\frac{\hbar}{2} \]

Characteristic equation; roots are real as the theorem guarantees. A

\[ |{+}\rangle_x=\frac{1}{\sqrt2}\begin{pmatrix}1\\1\end{pmatrix},\quad |{-}\rangle_x=\frac{1}{\sqrt2}\begin{pmatrix}1\\-1\end{pmatrix},\quad {}_x\langle{+}|{-}\rangle_x=0 \]

Solve \((\hat{S}_x-\lambda\hat{\mathbb{1}})v=0\) for each root, normalise; check orthogonality (step 8 of the derivation). B

\[ \hat{P}_{+}=\tfrac12\begin{pmatrix}1&1\\1&1\end{pmatrix},\quad \hat{P}_{-}=\tfrac12\begin{pmatrix}1&-1\\-1&1\end{pmatrix},\quad \hat{P}_++\hat{P}_-=\hat{\mathbb{1}} \]

Form the outer products; verify completeness and \(\hat{S}_x=\tfrac{\hbar}{2}\hat{P}_+-\tfrac{\hbar}{2}\hat{P}_-\). B

\[ \hat{S}_x=\frac{\hbar}{2}\,\hat{P}_{+}-\frac{\hbar}{2}\,\hat{P}_{-},\qquad \text{measuring a spin }|{\uparrow}\rangle_z=\begin{pmatrix}1\\0\end{pmatrix}\text{ gives }P(\pm)=\tfrac12 \]

Reading. A \(z\)-up spin measured along \(x\) yields \(+\hbar/2\) or \(-\hbar/2\) each with probability \(\lVert\hat{P}_\pm(1,0)^{\mathsf T}\rVert^2=\tfrac12\). Numerically \(a_\pm=\pm\,5.27\times10^{-35}\ \mathrm{J\,s}\) (using \(\hbar/2=5.27\times10^{-35}\ \mathrm{J\,s}\)).

\[ \hat{H}=\varepsilon\begin{pmatrix}2&0&0\\0&1&i\\0&-i&1\end{pmatrix},\qquad \varepsilon=1\ \mathrm{eV} \]

A three-level Hamiltonian; \(\hat{H}^{\dagger}=\hat{H}\) since the off-diagonal pair is \(i,-i\) (conjugates). Look for a degenerate block. A

\[ \det(\hat{H}-E\hat{\mathbb{1}})=0:\quad E_1=2\varepsilon,\quad \text{and } (1\!-\!\tilde E)^2-1=0\Rightarrow \tilde E=0,2 \]

Block-diagonal: the \(1\times1\) gives \(2\varepsilon\); the lower \(2\times2\) gives eigenvalues \(0\) and \(2\varepsilon\). All real. B

\[ E=2\varepsilon\ (\text{degenerate},\,g=2),\qquad E=0\ (g=1) \]

The eigenvalue \(2\varepsilon\) appears twice — a degenerate eigenspace requiring Gram–Schmidt. B

\[ |E{=}0\rangle=\frac{1}{\sqrt2}\begin{pmatrix}0\\1\\ i\end{pmatrix},\quad |E{=}2\varepsilon\rangle_a=\begin{pmatrix}1\\0\\0\end{pmatrix},\quad |E{=}2\varepsilon\rangle_b=\frac{1}{\sqrt2}\begin{pmatrix}0\\1\\-i\end{pmatrix} \]

Solve each eigenspace; within the degenerate block choose an orthonormal pair. Check \(\langle E{=}0|E{=}2\varepsilon\rangle_b=\tfrac12(1-i\cdot i)\)… correct to \(\tfrac12(1-1)=0\). C

\[ \hat{H}=0\cdot\hat{P}_0+2\varepsilon\,\hat{P}_{2\varepsilon},\qquad \hat{P}_{2\varepsilon}=|{\cdots}\rangle_a\langle{\cdots}|_a+|{\cdots}\rangle_b\langle{\cdots}|_b \]

Reading. The spectrum is \(\{0,\,2\ \mathrm{eV}\}\) with the excited level doubly degenerate; the rank-2 projector \(\hat{P}_{2\varepsilon}\) collects both eigenvectors. A state prepared in the degenerate block returns \(2\ \mathrm{eV}=3.20\times10^{-19}\ \mathrm{J}\) with certainty regardless of its rotation within that block — the residual \(U(2)\) symmetry.

Units check. \(E_n\) in eV (energy); projectors dimensionless; \(\hat{H}\) carries eV throughout. \(2\ \mathrm{eV}=3.20\times10^{-19}\ \mathrm{J}\). Consistent.

Problems

Show directly that the eigenvalues of the Pauli matrix \(\hat{\sigma}_y=\begin{pmatrix}0&-i\\ i&0\end{pmatrix}\) are real, and find its normalised eigenvectors and the two projectors.
Solution
Characteristic equation \(\lambda^2-1=0\Rightarrow\lambda=\pm1\) (real, as \(\hat\sigma_y^\dagger=\hat\sigma_y\)). For \(\lambda=+1\): \((-i)v_2=v_1\) gives \(v_{+}=\tfrac{1}{\sqrt2}(1,i)^{\mathsf T}\). For \(\lambda=-1\): \(v_{-}=\tfrac{1}{\sqrt2}(1,-i)^{\mathsf T}\). Check \(\langle v_+|v_-\rangle=\tfrac12(1\cdot1+\overline{i}(-i))=\tfrac12(1-1)=0\). Projectors \(\hat P_{\pm}=|v_\pm\rangle\langle v_\pm|=\tfrac12\begin{pmatrix}1&\mp i\\ \pm i&1\end{pmatrix}\); verify \(\hat P_++\hat P_-=\hat{\mathbb 1}\) and \(\hat\sigma_y=\hat P_+-\hat P_-\).
Prove that if \(\hat{A}=\hat{A}^{\dagger}\) then \(\langle\psi|\hat{A}|\psi\rangle\) is real for every \(|\psi\rangle\), not only eigenstates. State why this makes the expectation value a legitimate measurement average.
Solution
\(\overline{\langle\psi|\hat A|\psi\rangle}=\langle\psi|\hat A^\dagger|\psi\rangle=\langle\psi|\hat A|\psi\rangle\) using conjugate-symmetry and \(\hat A^\dagger=\hat A\); a number equal to its own conjugate is real. Expanding \(|\psi\rangle=\sum_n c_n|a_n\rangle\) gives \(\langle\hat A\rangle=\sum_n|c_n|^2 a_n\), a real convex combination of the (real) eigenvalues with Born weights \(|c_n|^2\) — exactly the average of outcomes.
Let \(\hat{A}\) and \(\hat{B}\) be Hermitian with \([\hat{A},\hat{B}]=0\). Show they share a common eigenbasis. (Assume the spectrum of \(\hat{A}\) may be degenerate.)
Solution
Diagonalise \(\hat A=\sum_n a_n\hat P_n\). Since \([\hat A,\hat B]=0\), \(\hat B\) commutes with each \(\hat P_n\) (as \(\hat P_n\) is a polynomial in \(\hat A\)), so \(\hat B\) maps each eigenspace \(\mathrm{ran}\,\hat P_n\) into itself. Restricted to that eigenspace \(\hat B\) is still Hermitian, so by the spectral theorem it has an orthonormal eigenbasis there. Those bases (one per block) together diagonalise both operators simultaneously, giving joint eigenvectors \(|a_n,b_k\rangle\).
A qubit is in state \(|\psi\rangle=\cos\tfrac{\theta}{2}\,|0\rangle+\sin\tfrac{\theta}{2}\,|1\rangle\). Using the spectral decomposition of \(\hat{\sigma}_z=\mathrm{diag}(1,-1)\), compute the probabilities of the two outcomes and \(\langle\hat{\sigma}_z\rangle\) for \(\theta=60^{\circ}\).
Solution
\(\hat\sigma_z=(+1)|0\rangle\langle0|+(-1)|1\rangle\langle1|\). \(P(+1)=|\langle0|\psi\rangle|^2=\cos^2\tfrac\theta2\), \(P(-1)=\sin^2\tfrac\theta2\). At \(\theta=60^\circ\): \(\tfrac\theta2=30^\circ\), \(\cos^2 30^\circ=3/4\), \(\sin^2 30^\circ=1/4\). So \(P(+1)=0.75\), \(P(-1)=0.25\), and \(\langle\hat\sigma_z\rangle=(+1)(0.75)+(-1)(0.25)=0.5=\cos\theta=\cos60^\circ\). Check: \(\cos60^\circ=0.5\). Consistent.
Give an explicit \(2\times2\) matrix that is symmetric-looking but whose eigenvectors are not orthogonal, and identify which hypothesis of the spectral theorem it violates.
Solution
Take \(\hat M=\begin{pmatrix}1&1\\0&2\end{pmatrix}\). It is real but not symmetric: \(\hat M^{\mathsf T}\neq\hat M\), hence not Hermitian. Eigenvalues \(1,2\); eigenvectors \((1,0)^{\mathsf T}\) and \((1,1)^{\mathsf T}\), whose overlap is \(1\neq0\). The theorem fails because self-adjointness (step 3) is violated — orthogonality (step 8) relied on moving \(\hat A\) across the inner product, which is illegal here. Any non-normal matrix, \([\hat M,\hat M^\dagger]\neq0\), can defeat orthogonality or even completeness.