Probability
The Analysis of Data, volume 1
Continuous Mapping Theorem
$
\def\P{\mathsf{\sf P}}
\def\E{\mathsf{\sf E}}
\def\Var{\mathsf{\sf Var}}
\def\Cov{\mathsf{\sf Cov}}
\def\std{\mathsf{\sf std}}
\def\Cor{\mathsf{\sf Cor}}
\def\R{\mathbb{R}}
\def\c{\,|\,}
\def\bb{\boldsymbol}
\def\diag{\mathsf{\sf diag}}
\def\defeq{\stackrel{\tiny\text{def}}{=}}
\newcommand{\toop}{\xrightarrow{\scriptsize{\text{p}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tood}{\rightsquigarrow}
\newcommand{\iid}{\mbox{$\;\stackrel{\mbox{\tiny iid}}{\sim}\;$}}$
8.10. Continuous Mapping Theorem
A well known property of continuous functions is that they preserve limits. In other words, if $f$ is continuous and $a_n\to a$ then $f(a_n)\to f(a)$. The following proposition shows that a similar result applies to the three different modes of stochastic convergence.
Definition 8.10.1
(Continuous Mapping Theorem). Let $\bb f:\R^d\to\R^m$ be a function for which $\P(\bb f({\bb X})\text{ is continuous})=1$. Then,
\begin{align*}
{\bb X}^{(n)}&\toop \bb X \qquad \text{implies} \qquad \bb f({\bb X}^{(n)})\toop \bb f(\bb X)\\
{\bb X}^{(n)}&\tood \bb X \qquad \text{implies} \qquad \bb f({\bb X}^{(n)})\tood \bb f(\bb X)\\
{\bb X}^{(n)}&\tooas \bb X \qquad \text{implies} \qquad \bb f({\bb X}^{(n)})\tooas \bb f(\bb X).
\end{align*}
Proof.
We prove the first statement using Proposition 8.2.4. It is sufficient to show that for every sequence $n_1,n_2,\ldots$ we have a subsequence $m_1,m_2,\ldots$ along which $\bb f({\bb X}^{(m_i)})\toop \bb f(\bb X)$. A second use of Proposition 8.2.4, shows that we can find a subsequence $m_1,m_2,\ldots$ of $n_1,n_2,\ldots$ along which ${\bb X}^{(n)}$ converges to $\bb X$ with probability 1. Since continuous functions preserve limits this implies that $\bb f({\bb X}^{(n)})$ converges to $\bb f(\bb X)$ along that subsequence with probability 1, and the first statement follows.
We prove the second statement using the portmanteau theorem. It is sufficient to show that for a bounded and continuous function $h$, we have $\E(h(\bb f({\bb X}^{(n)})))\to \E(h(f({\bb X})))$. Since $\bb f,h$ are continuous with probability 1 and $\bb h$ is bounded, the function $g=h\circ \bb f$ is also continuous with probability 1 and bounded. It follows from the portmanteau theorem that $\E(g({\bb X}^{(n)}))\to \E(g({\bb X}))$, proving the second statement.
To prove the third statement, note that we have with probability 1 a continuous function of a convergent sequence. Using the fact that continuous functions preserve limits, we have convergence to the required limit with probability 1.
Example 8.10.1.
If ${\bb X}^{(n)} \toop {\bb X}$ and ${\bb Y}^{(n)} \toop {\bb Y}$, then
- ${\bb X}^{(n)}+{\bb Y}^{(n)}\toop {\bb X}+{\bb Y}$,
- ${\bb X}^{(n)}{ Y}^{(n)}_1\toop {\bb X}{ Y}_1$,
- ${\bb X}^{(n)}/{ Y}^{(n)}_1\toop {\bb X}/{Y}_1$ provided that the denominators are not zero.