Characteristic Functions and Moments

Derivation

Characteristic Functions and Moments

D-117 Home PU-106 Threads chance Depends on Expectation, Variance and Their Algebra

Statement

For a real random variable X the characteristic function is φ_X(t) = E[e^itX], defined for every t ∈ ℝ. We derive three properties: (i) φ_X(0) = 1 and |φ_X(t)| ≤ 1; (ii) if E[|X|ⁿ] < ∞ then φ_X is n-times differentiable and its derivatives at the origin generate the moments, φ_X^(k)(0) = i^k E[X^k] for k ≤ n; (iii) for independent X, Y the characteristic function factorises, φ_X+Y(t) = φ_X(t)φ_Y(t).

Why it matters

The characteristic function is the Fourier transform of a probability distribution, and it always exists — unlike the moment generating function E[e^tX], which can diverge for heavy-tailed laws such as the log-normal or Cauchy. Because |e^itX| = 1, the defining integral converges absolutely for every distribution whatsoever, making φ_X a universally available fingerprint of a law.

Its two structural properties — that differentiation at zero peels off moments, and that it turns convolution (addition of independent variables) into multiplication — make it the workhorse behind the Central Limit Theorem, the analysis of sums and random walks, and the uniqueness and continuity theorems that let us prove distributions converge. It converts hard integral problems about densities into easy algebraic ones about products.

Assumptions

X is a real-valued random variable on a probability space.Without a well-defined law the expectation E[e^itX] has no meaning; the entire construction is the expectation of a bounded complex function of X.

For the moment relations, the absolute moment E[|X|ⁿ] is finite.If E[|X|ⁿ]=∞ the n-th derivative of φ_X at t=0 need not exist, and no finite n-th moment can be read off. Existence of the moment is exactly what licenses differentiating under the expectation.

For factorisation, X and Y are independent.Independence is what makes E[e^itXe^itY] = E[e^itX]E[e^itY]. Drop it and the joint expectation carries covariance-type cross terms; the product rule fails and the transform of the sum is not the product.

Differentiation under the integral is justified by a dominating function.The interchange of d/dt and E[·] is not free; it needs an integrable dominator (here |X|^k). Without domination the limit defining the derivative cannot be moved inside the expectation.

Derivation

φ_X(t) = E[e^itX] = E[cos(tX)] + i E[sin(tX)]

Definition; Euler's formula splits the complex integrand into real and imaginary parts, both bounded by 1 in modulus so both expectations exist. A

φ_X(0) = E[e⁰] = E[1] = 1

Substitute t=0; the expectation of the constant 1 is 1 because the total probability mass is 1. A

|φ_X(t)| = |E[e^itX]| ≤ E[|e^itX|] = E[1] = 1

Jensen / triangle inequality for expectations: the modulus of a mean is at most the mean of the modulus, and |e^iθ|=1 for real θ. B

φ_X(t+h) − φ_X(t) = E[e^itX(e^ihX−1)]

Difference quotient setup; factor the common e^itX out of the two exponentials to isolate the increment. B

|e^ihX − 1| ≤ |hX|, so |e^itX(e^ihX−1)/h| ≤ |X|

The chord bound |e^iθ−1| ≤ |θ| supplies a dominating function |X|, which is integrable when E[|X|]<∞. This is the hypothesis of the Dominated Convergence Theorem. C

φ_X′(t) = lim_h→0 E[e^itX(e^ihX−1)/h] = E[iX e^itX]

Dominated convergence lets the limit pass inside the expectation; the pointwise limit of the bracket is iX e^itX. C

φ_X^(k)(t) = E[(iX)^k e^itX], k = 1,…,n

Iterate step 6; each differentiation is dominated by |X|^k, integrable since E[|X|ⁿ]<∞ forces all lower absolute moments finite. B

φ_X^(k)(0) = E[(iX)^k] = i^k E[X^k]

Set t=0 so e^itX=1; pull the constant i^k out. This is the moment-generation identity. A

φ_X(t) = ∑_k=0ⁿ (it)^k/k! · E[X^k] + o(tⁿ)

Taylor expand about t=0 using steps 2 and 8 for the coefficients; valid to order n when the n-th moment exists. B

φ_X+Y(t) = E[e^it(X+Y)] = E[e^itX e^itY]

Definition applied to the sum; exponent of a sum splits into a product of exponentials. A

= E[e^itX] · E[e^itY] = φ_X(t) φ_Y(t)

Independence of X and Y makes e^itX and e^itY independent, so the expectation of the product factorises. B

Result

φ_X(t) = E[e^itX], φ_X^(k)(0) = i^k E[X^k], φ_X+Y(t) = φ_X(t) φ_Y(t)

Reading. The characteristic function packages the whole law of X into one bounded function of a real dial t. Turning the dial slowly near zero and reading off successive slopes recovers the moments one at a time: the first derivative gives i times the mean, the second gives −1 times the mean square, and so on. And because it is a Fourier transform, adding independent randomness — which convolves densities — simply multiplies the transforms.

Units check. The exponent tX must be dimensionless, so t carries units reciprocal to X (if X is in metres, t is in per-metre). Then φ_X is dimensionless. Each derivative d/dt multiplies by a unit of X, and indeed φ_X^(k)(0)=i^kE[X^k] carries units of X^k, matching the k-th moment. Consistent.

Limiting cases

Degenerate (X = c constant): φ_X(t) = e^itc, a pure phase of unit modulus; every derivative at 0 gives (ic)^k, i.e. moment c^k.
Shift and scale: φ_aX+b(t) = e^itb φ_X(at) — centring subtracts a linear phase, scaling stretches the t-axis.
Sum of n i.i.d. copies: φ_{S_n}(t) = φ_X(t)ⁿ; the root of the CLT, since [φ(t/√n)]ⁿ → e^−t²σ²/2.
Standard normal: φ(t) = e^−t²/2; even, real, and its own transform up to scaling — the fixed point of the CLT flow.
Symmetric law (X and −X same law): φ_X is real and even; all odd moments and odd derivatives at 0 vanish.

Breaks when

Heavy tails with no finite moment: the Cauchy law has φ(t)=e^−|t|, which has no derivative at t=0. The moment-generation identity fails entirely because E[|X|]=∞ — there is no mean to read off.
Dependence between the summands: if X and Y are correlated the factorisation φ_X+Y=φ_Xφ_Y is false; e.g. Y=−X gives φ_X+Y(t)=1 while φ_X(t)φ_Y(t)=φ_X(t)φ_X(−t)=|φ_X(t)|²≠1 in general.
Only the k-th moment infinite: if E[|X|^k]=∞ but lower moments are finite, φ_X is only (k−1)-times differentiable; the Taylor expansion truncates and the k-th coefficient is not i^kE[X^k].
Reading moments off any t ≠ 0: φ_X^(k)(t)=E[(iX)^ke^itX] mixes moments with oscillation for t≠0; the clean identity i^kE[X^k] holds only at the origin.

Failure modes

Dropping the factor of i: writing φ^(k)(0)=E[X^k] instead of i^kE[X^k]. This is the MGF rule, not the characteristic-function rule; the i^k is essential and gives, e.g., φ″(0)=−E[X²] (note the minus sign).
Confusing φ with the MGF: assuming φ_X(t)=M_X(t). They are related by φ_X(t)=M_X(it) only when the MGF exists; for Cauchy the MGF does not exist at all yet φ is perfectly fine.
Assuming factorisation implies independence: φ_X+Y=φ_Xφ_Y can hold for dependent variables (it only says the sum has that distribution); independence is sufficient, not necessary, so one cannot infer independence from a product characteristic function.
Differentiating under the expectation without checking a moment exists: blindly writing φ′(0)=iE[X] for the Cauchy law — the step is illegal because there is no dominating integrable function and no finite mean.
Believing a real φ means a real (non-random) X: φ real means the law is symmetric about 0, not that X is deterministic.

Discussion

The characteristic function is precisely the Fourier transform of the probability measure, φ_X(t)=∫ e^itx dF(x). This single identification explains all three properties at once. Boundedness is because we integrate a unit-modulus kernel against a probability (total-mass-one) measure. Moment generation is the standard fact that differentiating a Fourier transform brings down a factor of the variable. Factorisation is the convolution theorem: the density of a sum of independent variables is the convolution of their densities, and Fourier transforms convert convolution into pointwise multiplication.

Two deep theorems make the transform genuinely useful rather than merely elegant. Lévy's uniqueness theorem says φ_X determines the law of X completely — distinct distributions have distinct characteristic functions — so the fingerprint is faithful. Lévy's continuity theorem says pointwise convergence of characteristic functions (to a function continuous at 0) is equivalent to convergence in distribution. Together these turn the Central Limit Theorem into a one-line computation: standardise, factorise into an n-th power, Taylor expand to second order using the moment identity, and watch [1−t²/2n+o(1/n)]ⁿ→e^−t²/2, the Gaussian characteristic function.

The subtlety students most often miss is the asymmetry between differentiability and moments. A finite 2m-th moment guarantees φ is 2m-times differentiable at 0, but the converse is one-sided: existence of the 2m-th derivative at 0 implies the finite 2m-th moment (for even orders), yet existence of an odd derivative does not guarantee the corresponding odd moment. Smoothness of φ at the origin controls tail decay of the distribution, while smoothness of the distribution (its density) controls tail decay of φ at infinity — a Fourier duality between local behaviour on one side and global behaviour on the other. This is why the Cauchy characteristic function e^−|t| has a corner at 0 (no moments) yet decays at infinity (bounded density).

Common misconceptions. The characteristic function is not a probability and can be complex and negative in its real part; only its value at 0 is pinned to 1. It is not the same as the moment generating function, and its universal existence is exactly the reason probabilists prefer it. Finally, moments do not always determine the law: two different distributions (e.g. certain log-normal variants) can share every moment, yet they must still have different characteristic functions — because the Taylor series of φ need not converge to φ, moment equality does not force φ equality.

Worked examples

Example 1 — Poisson mean and variance from φ.

X ~ Poisson(λ), φ_X(t) = ∑_k≥0 e^−λλ^k/k! · e^itk = e^{λ(e^it−1)}

Sum the exponential series with ratio λe^it. A

φ′(t) = e^{λ(e^it−1)} · λ i e^it ⇒ φ′(0) = iλ

Chain rule; at t=0 the exponential prefactor is 1. Then E[X]=φ′(0)/i=λ. B

φ″(0) = (iλ)² + i²λ = −λ² − λ ⇒ E[X²] = λ²+λ

Differentiate again (product of prefactor and λie^it); use E[X²]=φ″(0)/i². B

Var(X) = E[X²] − E[X]² = (λ²+λ) − λ² = λ

Definition of variance from moments. Numerically, if λ=3: mean 3, variance 3. A

E[X] = λ = 3, Var(X) = λ = 3

Reading. The Poisson characteristic function generates the familiar mean-equals-variance identity directly from two derivatives at the origin. Units: X is a dimensionless count, so all moments are pure numbers.

Example 2 — Sum of two independent Poissons via factorisation.

X ~ Poisson(2), Y ~ Poisson(5), independent

Given data; independence licenses the product rule. A

φ_X+Y(t) = φ_X(t)φ_Y(t) = e^{2(e^it−1)} · e^{5(e^it−1)}

Factorisation for independent sums; substitute each Poisson characteristic function. B

= e^{(2+5)(e^it−1)} = e^{7(e^it−1)}

Add exponents. This is exactly the characteristic function of Poisson(7). A

⇒ X+Y ~ Poisson(7), E[X+Y]=7, Var(X+Y)=7

Uniqueness theorem: matching characteristic function forces the law. Mean and variance follow from Example 1. B

X + Y ~ Poisson(λ₁+λ₂) = Poisson(7)

Reading. The Poisson family is closed under independent addition, and the characteristic function proves it in three lines by turning convolution into a product. Units: counts add to counts; λ values in events add directly.

Problems

(A) Compute the characteristic function of a Bernoulli(p) variable and read off its mean and variance.
Solution
φ(t)=E[e^itX]=(1−p)e⁰+p e^it=1−p+pe^it. Then φ′(t)=ip e^it, so φ′(0)=ip and E[X]=p. Next φ″(t)=i²p e^it=−pe^it, so φ″(0)=−p and E[X²]=p. Variance = p−p²=p(1−p).
(A) A variable has φ(t)=e^−t²/2. Find its mean and variance.
Solution
φ′(t)=−t e^−t²/2, so φ′(0)=0 giving E[X]=0/i=0. φ″(t)=(t²−1)e^−t²/2, so φ″(0)=−1 and E[X²]=−1/i²=1. Hence mean 0, variance 1−0=1: this is the standard normal.
(B) Let X be exponential with rate λ, density λe^−λx on x≥0. Show φ(t)=λ/(λ−it) and use it to get E[X] and E[X²].
Solution
φ(t)=∫₀^∞ e^itxλe^−λxdx=λ∫₀^∞e^{−(λ−it)x}dx=λ/(λ−it). Write φ(t)=λ(λ−it)⁻¹. Then φ′(t)=λ·i(λ−it)⁻², so φ′(0)=iλ/λ²=i/λ and E[X]=1/λ. Next φ″(t)=λ·2i²(λ−it)⁻³=−2λ(λ−it)⁻³, so φ″(0)=−2λ/λ³=−2/λ², giving E[X²]=−(−2/λ²)=2/λ². Variance =2/λ²−1/λ²=1/λ².
(B) If X₁,…,X_n are i.i.d. with characteristic function φ and mean μ, variance σ², find the characteristic function of the standardised sum Z_n=(S_n−nμ)/(σ√n) and its n→∞ limit.
Solution
Let Y_i=(X_i−μ)/(σ√n), so Z_n=∑Y_i. Each Y_i has mean 0 and variance 1/n. By factorisation φ_{Z_n}(t)=[ψ(t/(σ√n))]ⁿ where ψ is the characteristic function of X−μ. Taylor to second order: ψ(s)=1+i·0·s−½σ²s²+o(s²). So ψ(t/(σ√n))=1−t²/(2n)+o(1/n), and [1−t²/(2n)+o(1/n)]ⁿ→e^−t²/2. The limit is the standard-normal characteristic function; by Lévy continuity, Z_n→N(0,1) in distribution — the Central Limit Theorem.
(C) The Cauchy law has φ(t)=e^−|t|. Show it is stable: the average of n i.i.d. Cauchy variables is again Cauchy with the same law. Comment on why the moment-generation rule cannot be applied.
Solution
For S_n=X₁+…+X_n, factorisation gives φ_{S_n}(t)=(e^−|t|)ⁿ=e^−n|t|. The sample mean M_n=S_n/n has φ_{M_n}(t)=φ_{S_n}(t/n)=e^−n|t/n|=e^−|t|, identical to a single Cauchy. So averaging does not concentrate — the mean of n Cauchy variables has exactly the same spread as one. The moment rule fails because φ(t)=e^−|t| has a corner at t=0 (left derivative +1, right derivative −1): it is not differentiable there, reflecting E[|X|]=∞. There is no finite mean to generate, which is also why the Law of Large Numbers and CLT both fail for Cauchy.