The Boltzmann distribution from maximum entropy

Derivation · the atom

The Boltzmann distribution from maximum entropy

D-013 boltzmann-distribution Home PU-203 Threads chance · energy Depends on — verified

Statement

The probability distribution over states that maximises entropy at fixed mean energy is exponential in the energy, and the exponent defines temperature.

Why it matters

The Boltzmann factor is the single most-used result in physical science. Presented as an assertion it is a magic trick; derived, it is a statement about counting and nothing more. This route also makes clear that temperature is not a primitive concept — it is a Lagrange multiplier.

Assumptions

All microstates of the isolated total system are equally likely.The fundamental postulate. It is a postulate, not a theorem; attempts to derive it from dynamics (ergodicity) succeed only for restricted classes of system.

The system is weakly coupled to a much larger reservoir.Interaction energy negligible compared with either subsystem's energy. Fails for small systems and for anything with long-range forces, where surface terms are not negligible.

Only the mean energy is constrained.Adding a constraint on particle number produces the grand canonical distribution and a second multiplier, the chemical potential.

Derivation

S = −k_B Σ_i p_i ln p_i

The Gibbs entropy. For a uniform distribution over Ω states it reduces to k_B ln Ω, recovering Boltzmann's expression, so this is a generalisation rather than a new assumption. A

Σ_i p_i = 1 , Σ_i p_i E_i = U

The two constraints. Normalisation, and a fixed mean energy — the latter is what "in contact with a reservoir at fixed temperature" amounts to. A

δ [ S − α k_B Σ p_i − β k_B Σ p_i E_i ] = 0

Maximise S subject to both constraints by the method of Lagrange multipliers. The multipliers α and β are undetermined for now; their meaning emerges rather than being imposed. B

−ln p_i − 1 − α − β E_i = 0

Differentiate with respect to each p_i independently and set to zero. Because the p_i are now unconstrained (the constraints having been absorbed into the multipliers), each derivative vanishes separately. B

p_i = e^−1−α e^−βE_i ≡ (1/Z) e^−βE_i

Exponentiate. The first factor is a constant, fixed by normalisation, and is renamed 1/Z. The partition function is therefore not a definition imposed from outside — it is whatever makes the probabilities sum to one. B

∂S/∂U = k_B β ≡ 1/T ⟹ β = 1/(k_BT)

Substitute the solution back into S and differentiate with respect to U. Comparing with the thermodynamic definition of temperature identifies the multiplier. Temperature has arrived as a derived quantity. C

Result

p_i = e^−E_i/k_BT / Z , Z = Σ_i e^−E_i/k_BT

Reading. States of higher energy are exponentially less likely, with k_BT setting the scale over which "less likely" becomes "essentially never". Every thermodynamic quantity follows from Z: U = −∂ln Z/∂β, F = −k_BT ln Z, and so on.

Units check. The exponent is dimensionless: [J] / ([J K⁻¹][K]). ✓

Limiting cases

Uniform distribution over Ω equally-likely states recovers Boltzmann's S = k_B ln Ω.
High temperature (k_BT ≫ level spacing): all accessible states become nearly equiprobable.
Low temperature (k_BT ≪ level spacing): the system collapses onto the ground state.

Breaks when

The system is small enough that its energy fluctuations are comparable to its mean — the ensembles are then inequivalent.
Long-range interactions make energy non-additive, so the reservoir decomposition fails. Self-gravitating systems are the standard pathology, and they can exhibit negative heat capacity.
The system is not in equilibrium. Nothing here applies to a driven or relaxing system.

Failure modes

Confusing states with energy levels. A level with degeneracy g contributes g e^−βE, and forgetting the degeneracy is the most common numerical error in the topic.
Treating Z as a mere normalisation. It encodes the complete thermodynamics; the normalisation role is incidental.
Assuming the highest-probability state is where the system is found. With degeneracy the most probable energy can be far above the ground state — see D-014, where the peak of the speed distribution is not at zero.

Worked number

The ratio of populations of two levels is p₂/p₁ = e^−ΔE/k_BT, with Z cancelling. At 300 K, k_BT = 0.0259 eV. For the CO vibrational spacing of 0.27 eV from D-010, the ratio is e^−10.4 ≈ 3 × 10⁻⁵ — three molecules in a hundred thousand are vibrationally excited. This one line explains the frozen-out heat capacity of D-020.

Run the check →

Discussion

The single most important thing to see is that the exponential form is not chosen — it is forced. When a large isolated composite (system + reservoir) is free to share a fixed total energy, the fundamental postulate weights every microstate equally. Overwhelmingly the most microstates correspond to the reservoir absorbing energy E_i whenever the small system sits in state i, and the number of reservoir microstates grows as e^{S_R(U−E_i)/k_B}. Taylor-expanding the reservoir entropy to first order in the small quantity E_i produces exactly e^−E_i/k_BT. The max-entropy route in the derivation and this counting route are the same statement dressed differently: the exponential is what you get when the only thing you know is the mean energy.

The reason a single scalar function Z carries the entire thermodynamics is that the energy constraint entered the exponent linearly. That makes Z(β) a generating function: each derivative with respect to β pulls down another power of E. Hence U = −∂lnZ/∂β is the mean, and the second derivative gives the variance, ∂²lnZ/∂β² = ⟨E²⟩ − ⟨E⟩² = k_BT²C_V. That last equality is a fluctuation–response relation of exactly the type that recurs across physics: the equilibrium fluctuations of a quantity and the system's response to a conjugate perturbation are one and the same object. The free energy F = −k_BT lnZ then makes Z the bridge between microscopic spectrum and macroscopic thermodynamics.

Temperature appearing as a Lagrange multiplier is not a curiosity of this derivation — it is the deepest content. In Jaynes' information-theoretic reading, the Boltzmann distribution is simply the least-committal probability assignment (maximum Shannon/Gibbs entropy) consistent with a known mean, and β is the multiplier enforcing that mean. The pair (S as a function of U) and (lnZ as a function of β) are Legendre transforms of one another, the same duality that links Lagrangian and Hamiltonian, or internal energy and free energy. This viewpoint also predicts its own generalisations without new ideas: add a number constraint and you get a second multiplier μ/k_BT and the grand canonical distribution; constrain volume-conjugate or magnetisation-conjugate quantities and you get the corresponding ensembles. It even permits T < 0: if the energy spectrum is bounded above (as for a fixed set of spins), a distribution that favours high-energy states is still a legitimate maximiser, and the multiplier β simply goes negative — negative temperatures are hotter than infinite temperature, not colder than zero.

Common misconceptions. (i) That the Boltzmann factor gives the probability of an energy. It gives the probability of a state; the probability of an energy level of degeneracy g carries an extra factor g, which is why the most-populated level need not be the ground level (see the rotational example below). (ii) That Z is "just normalisation". Normalisation is incidental; Z encodes the full thermodynamics. (iii) That temperature is a primitive that causes the distribution. Here it is the other way round: fixing the mean energy defines a multiplier, and that multiplier is what we have always called 1/k_BT.

Worked examples

Example 1 — A single spin in a magnetic field (two-level system). An electron magnetic moment of magnitude μ = μ_B = 9.274×10⁻²⁴ J T⁻¹ sits in a field B = 2 T at T = 4 K. Find the population ratio, the fractional populations, and the mean energy per spin.

Z = e^+βμB + e^−βμB = 2 cosh(βμB)

Two states: aligned (E = −μB) and anti-aligned (E = +μB). The splitting is ΔE = 2μB. Keep everything symbolic first.

βμB = μB/k_BT = (9.274×10⁻²⁴ × 2)/(1.381×10⁻²³ × 4) = 0.336

Evaluate the dimensionless exponent. Only now do numbers enter.

p₊/p₋ = e^−2βμB = e^−0.672 = 0.511

The ratio of anti-aligned to aligned populations; Z cancels, as always for a ratio.

p₋ = e^+0.336/2cosh(0.336) = 0.662, p₊ = 0.338

Divide each Boltzmann factor by Z = 2.114. The two fractions sum to 1, as they must.

⟨E⟩ = μB(p₊ − p₋) = −μB tanh(βμB) = −(1.855×10⁻²³)(0.323)

The mean energy per spin. tanh(0.336) = 0.323 gives the polarisation directly.

⟨E⟩ = −μB tanh(μB/k_BT) = −6.0×10⁻²⁴ J

Answer. About 66% of spins align with the field and 34% oppose it, giving a mean energy of −6.0×10⁻²⁴ J per spin (a polarisation of only 0.32). Even at 4 K and 2 T the sample is far from fully aligned, because μB is still a fraction of k_BT — the origin of Curie's law, in which magnetisation scales as B/T in the weak-field regime.

Example 2 — The most-populated rotational level of CO (degeneracy matters). A linear rotor has levels E_J = B_rotJ(J+1) with degeneracy g_J = 2J+1. For CO the rotational constant is B̃ = 1.93 cm⁻¹. Which level is most heavily populated at 300 K?

P_J ∝ (2J+1) e^{−βB_rotJ(J+1)}

The population of a level is its degeneracy times the Boltzmann factor of a state in it. Dropping the 2J+1 is the classic error flagged in the failure modes.

d/dJ [ ln(2J+1) − βB_rotJ(J+1) ] = 0 ⟹ J_max = √(k_BT/2B_rot) − ½

Treat J as continuous and maximise the log-population. The competition is degeneracy (rising) against the Boltzmann factor (falling).

B_rot = hc×(1.93 cm⁻¹) = (1.986×10⁻²³ J·cm)(1.93 cm⁻¹) = 3.83×10⁻²³ J

Convert the spectroscopic constant to energy with hc. Here hc = 1.986×10⁻²³ J·cm.

k_BT = 4.14×10⁻²¹ J; J_max = √(4.14×10⁻²¹/(2×3.83×10⁻²³)) − ½ = 7.35 − 0.5 = 6.85

At T = 300 K. The nearest physical (integer) level is J = 7.

P₇/P₀ = 15 e^{−B_rot×56/k_BT} = 15 e^−0.518 = 8.9

Compare the peak level (g = 15, E = 56B_rot) with the ground level (g = 1, E = 0).

J_max = √(k_BT/2B_rot) − ½ ≈ 7

Answer. The most populated rotational level of CO at 300 K is J = 7, with P₇ ≈ 8.9 P₀ — nearly nine times the ground level, even though it lies 56B_rot higher. The peak sits well above the ground state purely because of the 2J+1 degeneracy, the same mechanism that shifts the peak of the Maxwell speed distribution away from zero (D-014).

Problems

(Warm-up) Two non-degenerate levels are split by ΔE = 0.10 eV. What fraction of systems occupies the upper level at 300 K, and at 1000 K? (k_BT = 0.0259 eV at 300 K.)
Solution
Ratio r = p_u/p_ℓ = e^−ΔE/k_BT. At 300 K: r = e^{−0.10/0.0259} = e^−3.86 = 0.0210, so upper fraction = r/(1+r) = 0.0206 (about 2%). At 1000 K: k_BT = 0.0259×(1000/300) = 0.0863 eV, r = e^−1.16 = 0.314, fraction = 0.239 (about 24%). Raising T flattens the distribution toward equal occupation.
(Two-level thermodynamics) For a two-level system with energies 0 and ε (both non-degenerate), write Z and ⟨E⟩, evaluate ⟨E⟩/ε when βε = 1, and give the limits T→0 and T→∞.
Solution
Z = 1 + e^−βε. Then ⟨E⟩ = ε e^−βε/(1+e^−βε) = ε/(e^βε+1). At βε = 1: ⟨E⟩ = ε/(e+1) = ε/3.718 = 0.269 ε. Limits: as T→0 (βε→∞), ⟨E⟩→0 (system frozen in ground state); as T→∞ (βε→0), ⟨E⟩→ε/2 (both states equally likely). A two-level system therefore stores at most ε/2 — it saturates, unlike an unbounded spectrum.
(Degeneracy) A system has three levels at energies 0, ε, 2ε with degeneracies 1, 2, 1. Take ε = 0.025 eV and a temperature such that k_BT = 0.025 eV (i.e. 290 K). Find Z and the probability of each level. Which single microstate is most probable?
Solution
Here βε = 1. Z = 1·e⁰ + 2·e⁻¹ + 1·e⁻² = 1 + 0.736 + 0.135 = 1.871. Level probabilities: P(0) = 1/1.871 = 0.534; P(ε) = 0.736/1.871 = 0.393; P(2ε) = 0.135/1.871 = 0.072. The middle level is boosted by its degeneracy, but per state the two states at ε each carry e⁻¹/Z = 0.197, less than the ground state's 0.534. So the single most probable microstate is the non-degenerate ground state — the distinction between "most probable level" and "most probable state" is the whole point.
(Schottky heat capacity) For the two-level system of problem 2, show that the heat capacity is C = k_B(βε)² e^βε/(e^βε+1)². It peaks at βε ≈ 2.40. For ε = 1.0 meV, at what temperature does the peak occur, and what is C_max?
Solution
With ⟨E⟩ = ε/(e^βε+1) and C = d⟨E⟩/dT, put x = βε = ε/k_BT so dx/dT = −x/T. Then d⟨E⟩/dx = −εe^x/(e^x+1)², giving C = (−εe^x/(e^x+1)²)(−x/T) = k_Bx² e^x/(e^x+1)² (using ε/T = xk_B). This is the Schottky anomaly: C→0 at both low and high T, with a bump in between. Setting dC/dx = 0 gives numerically x ≈ 2.40. At ε = 1.0 meV: T = ε/(k_Bx) = 1.0×10⁻³ eV/(8.617×10⁻⁵ eV K⁻¹ × 2.40) = 4.8 K. And C_max = k_B(2.40)²e^2.40/(e^2.40+1)² = k_B(5.76×11.02/144.5) = 0.44 k_B per system.
(Capstone — quantum oscillator) A quantum harmonic oscillator has levels E_n = nℏω, n = 0, 1, 2, … (measure from the ground state; non-degenerate). (a) Sum the geometric series for Z. (b) Obtain ⟨E⟩ = −∂lnZ/∂β. (c) Evaluate ⟨E⟩ for the CO stretch, ℏω = 0.27 eV, at 300 K, and explain the frozen-out heat capacity of D-020.
Solution
(a) Z = Σ_n≥0 e^−nβℏω = 1/(1 − e^−βℏω) (a geometric series with ratio e^−βℏω < 1). (b) lnZ = −ln(1 − e^−βℏω), so ⟨E⟩ = −∂lnZ/∂β = ℏω/(e^βℏω − 1) — the Planck–Einstein result. (Its high-T limit is k_BT, recovering equipartition for a 1D oscillator; its low-T limit is ℏωe^−βℏω → 0.) (c) βℏω = 0.27/0.0259 = 10.4, so e^10.4 = 3.35×10⁴ and ⟨E⟩ = 0.27/(3.35×10⁴ − 1) = 8.1×10⁻⁶ eV per molecule — utterly negligible next to k_BT = 0.026 eV. The mode is "frozen": it holds essentially no energy and its contribution to C_V is exponentially suppressed (≈ k_B(βℏω)²e^−βℏω), which is precisely why the vibrational heat capacity of CO does not switch on until temperatures of order ℏω/k_B ≈ 3100 K (D-020).