The Generalized Eigenvalue Problem and M-Orthogonality
~7 min read · Tier toggle changes depth (T1 plain · T2 standard · T3 rigour)
You should be comfortable with matrix algebra, the concept of a linear operator, and the standard eigenvalue problem \(\mathbf{A}\mathbf{v}=\lambda\mathbf{v}\). Familiarity with the derivation of normal modes from coupled oscillators (D-085) is assumed.
Why this matters
In physical systems, the "mass" or "metric" of the space is rarely the identity matrix.
When we analyze coupled oscillators or quantum systems in non-orthogonal bases, the standard Euclidean dot product fails to capture the correct geometric relationships. The generalized eigenvalue problem arises naturally when the kinetic energy (or norm) is defined by a matrix \(\mathbf{M}\) rather than the identity. Understanding \(M\)-orthogonality is essential because it dictates how modes decouple and how energy is distributed among them, forming the bridge between classical mechanics and the spectral theory of self-adjoint operators.
The Idea in Plain Language
Imagine you are trying to find the natural frequencies of a system of masses connected by springs. In the simplest case, all masses are equal, and the "distance" between configurations is measured by the standard Euclidean norm. However, if the masses are different, the "cost" of moving one coordinate is different from moving another. The matrix \(\mathbf{M}\) encodes these masses. When we look for the modes of vibration, we are not just looking for vectors that get stretched by the stiffness matrix \(\mathbf{K}\); we are looking for vectors that get stretched by \(\mathbf{K}\) relative to the "size" defined by \(\mathbf{M}\). Two modes are "orthogonal" in this context not if their standard dot product is zero, but if their interaction weighted by the masses is zero. This is \(M\)-orthogonality. It ensures that the modes are independent in the physical sense: energy put into one mode does not leak into another.
From Standard to Generalized Eigenvalues
The standard eigenvalue problem is \(\mathbf{K}\mathbf{a} = \lambda \mathbf{a}\). In the context of coupled oscillators, the equation of motion is \(\mathbf{M}\ddot{\mathbf{q}} + \mathbf{K}\mathbf{q} = \mathbf{0}\). Assuming a harmonic solution \(\mathbf{q}(t) = \mathbf{a} e^{i\omega t}\), we substitute to find \(-\omega^2 \mathbf{M}\mathbf{a} + \mathbf{K}\mathbf{a} = \mathbf{0}\), which rearranges to:
Here, \(\lambda = \omega^2\). This is the generalized eigenvalue problem. Unlike the standard case, the eigenvectors \(\mathbf{a}_j\) are not necessarily orthogonal with respect to the standard dot product \(\mathbf{a}_j^{\mathsf T}\mathbf{a}_k\). Instead, they are orthogonal with respect to the inner product defined by \(\mathbf{M}\).
Because \(\mathbf{M}\) is symmetric and positive-definite (it represents mass), and \(\mathbf{K}\) is symmetric (it represents stiffness), the eigenvalues \(\omega_k^2\) are guaranteed to be real and positive (assuming stability). The eigenvectors can be chosen such that they form an \(M\)-orthonormal basis:
This condition is crucial. It allows us to diagonalize the system completely. If we form the matrix \(\mathbf{P}\) whose columns are the eigenvectors \(\mathbf{a}_k\), then \(\mathbf{P}^{\mathsf T}\mathbf{M}\mathbf{P} = \mathbf{I}\) and \(\mathbf{P}^{\mathsf T}\mathbf{K}\mathbf{P} = \mathbf{\Lambda}\), where \(\mathbf{\Lambda}\) is the diagonal matrix of eigenvalues \(\omega_k^2\).
Why M-Orthogonality Holds
Why does \(\mathbf{a}_j^{\mathsf T}\mathbf{M}\mathbf{a}_k\) vanish for \(j \neq k\)? Let us consider two distinct eigenvalues \(\omega_j^2\) and \(\omega_k^2\) with corresponding eigenvectors \(\mathbf{a}_j\) and \(\mathbf{a}_k\). We have:
Multiply the first equation on the left by \(\mathbf{a}_k^{\mathsf T}\) and the second on the left by \(\mathbf{a}_j^{\mathsf T}\):
Since \(\mathbf{K}\) is symmetric, \(\mathbf{a}_k^{\mathsf T}\mathbf{K}\mathbf{a}_j = (\mathbf{a}_k^{\mathsf T}\mathbf{K}\mathbf{a}_j)^{\mathsf T} = \mathbf{a}_j^{\mathsf T}\mathbf{K}^{\mathsf T}\mathbf{a}_k = \mathbf{a}_j^{\mathsf T}\mathbf{K}\mathbf{a}_k\). Therefore, the left-hand sides are equal. Equating the right-hand sides:
Since \(\mathbf{M}\) is symmetric, \(\mathbf{a}_k^{\mathsf T}\mathbf{M}\mathbf{a}_j = \mathbf{a}_j^{\mathsf T}\mathbf{M}\mathbf{a}_k\). Let this scalar be \(S\). Then \((\omega_j^2 - \omega_k^2)S = 0\). If \(\omega_j^2 \neq \omega_k^2\), we must have \(S = 0\). Thus, eigenvectors corresponding to distinct eigenvalues are \(M\)-orthogonal. If eigenvalues are degenerate, we can choose a basis within the degenerate subspace that is \(M\)-orthogonal, leveraging the positive-definiteness of \(\mathbf{M}\).
Connection to Variational Principles
The generalized eigenvalue problem is not just an algebraic curiosity; it is the stationary point condition of a variational principle. The Rayleigh quotient for the generalized problem is defined as:
According to the variational characterization of eigenvalues (D-086), the stationary points of this quotient with respect to variations in \(\mathbf{a}\) are precisely the eigenvectors of the generalized problem, and the stationary values are the eigenvalues \(\omega_k^2\). The lowest eigenvalue \(\omega_1^2\) is the global minimum of \(R(\mathbf{a})\), and higher eigenvalues are obtained by imposing orthogonality constraints with respect to the lower modes. This provides a powerful method for estimating eigenvalues without solving the full matrix equation, particularly useful when \(\mathbf{M}\) and \(\mathbf{K}\) are large or sparse.
The derivations
Normal Modes from the Generalized Eigenvalue Problem
For a conservative mechanical system with \(n\) degrees of freedom executing small oscillations about a stable equilibrium, with kinetic energy \(T=\tfrac12\,\dot{\mathbf{q}}^{\mathsf T}\mathbf{M}\dot{\mathbf{q}}\) and potential energy \(V=\tfrac12\,\mathbf{q}^{\mathsf T}\mathbf{K}\mathbf{q}\) (both quadratic forms, \(\mathbf{M}\) symmetric positive-definite, \(\mathbf{K}\) symmetric), the linearized equations of motion \(\mathbf{M}\ddot{\mathbf{q}}+\mathbf{K}\mathbf{q}=\mathbf{0}\) admit harmonic solutions \(\mathbf{q}(t)=\mathbf{a}\,e^{i\omega t}\) precisely when the amplitude vector \(\mathbf{a}\) and frequency \(\omega\) solve the generalized eigenvalue problem \(\mathbf{K}\mathbf{a}=\omega^2\mathbf{M}\mathbf{a}\). The \(n\) eigenvalues \(\omega_k^2\) are real, and the eigenvectors can be chosen \(\mathbf{M}\)-orthonormal, \(\mathbf{a}_j^{\mathsf T}\mathbf{M}\mathbf{a}_k=\delta_{jk}\) and \(\mathbf{a}_j^{\mathsf T}\mathbf{K}\mathbf{a}_k=\omega_k^2\,\delta_{jk}\), so the normal modes simultaneously diagonalize both the inertia and the stiffness.
Variational Characterization of Eigenvalues
Let \(\hat{A}\) be a self-adjoint operator on a Hilbert space \(\mathcal{H}\) with a discrete, bounded-below spectrum and eigenvalues ordered \(\lambda_1 \le \lambda_2 \le \lambda_3 \le \cdots\). Define the Rayleigh quotient \(R(\psi) = \dfrac{\langle \psi, \hat{A}\psi\rangle}{\langle \psi,\psi\rangle}\) for \(\psi \neq 0\). Then the stationary points of \(R\) are exactly the eigenvectors of \(\hat{A}\), the stationary values are the eigenvalues, and the ordered eigenvalues obey the Courant–Fischer min–max principle \(\lambda_n = \min_{\dim S = n}\ \max_{\psi \in S \setminus \{0\}} R(\psi)\).
A common pitfall is assuming that \(M\)-orthogonality implies standard orthogonality. If \(\mathbf{M}\) is not the identity, \(\mathbf{a}_j^{\mathsf T}\mathbf{a}_k\) is generally non-zero even if \(\mathbf{a}_j^{\mathsf T}\mathbf{M}\mathbf{a}_k = 0\). When transforming coordinates to normal modes, you must use the inverse of the mass-weighted transformation matrix, not simply the transpose, to preserve the form of the kinetic energy. C
Worked example
Problem. Consider a two-degree-of-freedom system with mass matrix \(\mathbf{M} = \begin{pmatrix} 2 & 0 \\ 0 & 1 \end{pmatrix}\) and stiffness matrix \(\mathbf{K} = \begin{pmatrix} 3 & -1 \\ -1 & 2 \end{pmatrix}\). Find the eigenvalues \(\omega^2\) and the first eigenvector \(\mathbf{a}_1\) normalized such that \(\mathbf{a}_1^{\mathsf T}\mathbf{M}\mathbf{a}_1 = 1\).
- Set up the generalized eigenvalue problem: \(\det(\mathbf{K} - \lambda \mathbf{M}) = 0\), where \(\lambda = \omega^2\).
- Compute the determinant: \[ \det \begin{pmatrix} 3 - 2\lambda & -1 \\ -1 & 2 - \lambda \end{pmatrix} = (3 - 2\lambda)(2 - \lambda) - (-1)(-1) = 0 \] \[ 6 - 3\lambda - 4\lambda + 2\lambda^2 - 1 = 0 \implies 2\lambda^2 - 7\lambda + 5 = 0 \]
- Solve the quadratic equation: \[ \lambda = \frac{7 \pm \sqrt{49 - 40}}{4} = \frac{7 \pm 3}{4} \] Thus, \(\lambda_1 = 1\) and \(\lambda_2 = 2.5\).
- Find the eigenvector for \(\lambda_1 = 1\): \[ (\mathbf{K} - 1\mathbf{M})\mathbf{a}_1 = \begin{pmatrix} 3-2 & -1 \\ -1 & 2-1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \begin{pmatrix} 1 & -1 \\ -1 & 1 \end{pmatrix} \begin{pmatrix} x \\ y \end{pmatrix} = \mathbf{0} \] This gives \(x - y = 0 \implies x = y\). Let \(\mathbf{a}_1 = \begin{pmatrix} 1 \\ 1 \end{pmatrix}\).
- Normalize with respect to \(\mathbf{M}\): \[ \mathbf{a}_1^{\mathsf T}\mathbf{M}\mathbf{a}_1 = \begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} 2 & 0 \\ 0 & 1 \end{pmatrix} \begin{pmatrix} 1 \\ 1 \end{pmatrix} = \begin{pmatrix} 1 & 1 \end{pmatrix} \begin{pmatrix} 2 \\ 1 \end{pmatrix} = 3 \] To make this equal to 1, scale by \(1/\sqrt{3}\). \[ \mathbf{a}_1 = \frac{1}{\sqrt{3}} \begin{pmatrix} 1 \\ 1 \end{pmatrix} \]
Units check. \(\mathbf{K}\) has units of \(\mathrm{N/m}\) (or \(\mathrm{kg/s^2}\)), \(\mathbf{M}\) has units of \(\mathrm{kg}\). \(\lambda = \omega^2\) has units \(\mathrm{s^{-2}}\). The eigenvector is dimensionless. The normalization condition \(\mathbf{a}^{\mathsf T}\mathbf{M}\mathbf{a}\) has units of \(\mathrm{kg}\), which is consistent with the definition of the inner product in this space.
Three quick tests
- Why must the mass matrix \(\mathbf{M}\) be positive-definite for the generalized eigenvalue problem to have real, positive eigenvalues?
Answer
Positive-definiteness ensures that the "norm" \(\mathbf{a}^{\mathsf T}\mathbf{M}\mathbf{a}\) is always positive for non-zero \(\mathbf{a}\), allowing the Rayleigh quotient to be well-defined and bounded below. Combined with the symmetry of \(\mathbf{K}\), this guarantees real eigenvalues. - If \(\mathbf{M} = \mathbf{I}\), how does \(M\)-orthogonality relate to standard orthogonality?
Answer
They are identical. \(\mathbf{a}_j^{\mathsf T}\mathbf{I}\mathbf{a}_k = \mathbf{a}_j^{\mathsf T}\mathbf{a}_k\), which is the standard Euclidean dot product. - Can two eigenvectors corresponding to the same eigenvalue be \(M\)-orthogonal?
Answer
Yes. If the eigenvalue is degenerate, the eigenspace has dimension greater than 1. Since \(\mathbf{M}\) is positive-definite, it defines a valid inner product on this subspace, allowing us to choose an \(M\)-orthonormal basis for the degenerate subspace.
Problem set for PU-104 →
Each derivation in this unit ends with five graded problems with full solutions.
Lecture notes drafted with a local model and checked line by line by the editor; every number in the worked example was recomputed.
Derivations cited: D-085 · D-086 · Threads: symmetry, waves, matter