Probability
The Analysis of Data, volume 1
Random Variables: The Moment Generating Function
$
\def\P{\mathsf{\sf P}}
\def\E{\mathsf{\sf E}}
\def\Var{\mathsf{\sf Var}}
\def\Cov{\mathsf{\sf Cov}}
\def\std{\mathsf{\sf std}}
\def\Cor{\mathsf{\sf Cor}}
\def\R{\mathbb{R}}
\def\defeq{\stackrel{\tiny\text{def}}{=}}
\def\c{\,|\,}
\def\bb{\boldsymbol}
\def\diag{\mathsf{\sf diag}}
$
2.4. Moments and the Moment Generating Function
Definition 2.4.1.
The $k$-moment of an RV $X$ is $\E(X^k)$.
Definition 2.4.2.
The moment generating function (mgf) of the RV $X$ is the function
\[m:\mathbb{R}\to\mathbb{R},\qquad m(t) = \E(\exp(tX)).\]
The name mgf stems from the fact that its derivatives, evaluated at 0 produce the moments.
Proposition 2.4.1.
\[ m^{(k)}(0) = \E(X^k)\]
Proof.
Using the Taylor series expansion (Section D.2) of $e^{z}$ around 0, we have
\begin{align*}
m'(0) &= \frac{d}{dt}\E\left(1+tX+\frac{t^2X^2}{2!}+\cdots\right)\Big|_{t=0} =
\E\left(\frac{d}{dt}\left(1+tX+\frac{t^2X^2}{2!}+\cdots\right)\right)\Big|_{t=0}
\\&= \E(X).\end{align*}
Similarly, the second derivative produces the second
moment
\begin{align*}
m''(0) &=
\frac{d^2}{dt^2}\E\left(1+tX+\frac{t^2X^2}{2!}+\cdots\right)\Big|_{t=0}
\\ &=
\E\left(\frac{d^2}{dt^2}\left(1+tX+\frac{t^2X^2}{2!}+\cdots\right)\right)\Big|_{t=0}
\\ &= \E(X^2),
\end{align*}
and so on for the higher-order derivatives.
In addition to producing the moments of $X$, the mgf is useful in
identifying the distribution of $X$.
Proposition 2.4.2.
Let $X_1,X_2$ be two RVs with mgfs $m_1,m_2$. If $m_1(t) = m_2(t)$ for all
$t\in(-\epsilon,+\epsilon)$, for some $\epsilon>0$, then the two RVs
have identical cdfs (and therefore identical pdfs or pmfs).
Proof*.
According to a result from complex analysis, an analytic function (a function determined locally by its Taylor series) has a unique extension, called the analytics continuation, from $(-\epsilon,\epsilon)$ to the right half of the complex plane. Since the moment generating function $m(t)$ agrees with the characteristic function $\phi(-it)$ for $t\in\R$, and the characteristic function uniquely determines the distribution (see Section 8.7 for a description of the characteristic function and a proof that it uniquely determines the distribution), the moment generating function uniquely determines the distribution as well.
The moments of an RV $X$, provided they exist, completely characterize the Taylor series of the mgf and, therefore, characterize the mgf. Together with the above proposition, the moments $\E(X^k), k\in\mathbb{N}$ (assuming the corresponding integrals or sums converge) completely characterize the distribution of $X$ and the corresponding cdf, pdf, and pmf.