Probability
The Analysis of Data, volume 1
Slustky's Theorems
$
\def\P{\mathsf{\sf P}}
\def\E{\mathsf{\sf E}}
\def\Var{\mathsf{\sf Var}}
\def\Cov{\mathsf{\sf Cov}}
\def\std{\mathsf{\sf std}}
\def\Cor{\mathsf{\sf Cor}}
\def\R{\mathbb{R}}
\def\c{\,|\,}
\def\bb{\boldsymbol}
\def\diag{\mathsf{\sf diag}}
\def\defeq{\stackrel{\tiny\text{def}}{=}}
\newcommand{\toop}{\xrightarrow{\scriptsize{\text{p}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tooas}{\xrightarrow{\scriptsize{\text{as}}}}
\newcommand{\tood}{\rightsquigarrow}
\newcommand{\iid}{\mbox{$\;\stackrel{\mbox{\tiny iid}}{\sim}\;$}}$
8.11. Slustky's Theorems
Proposition 8.11.1 (Slutsky's Theorem).
\begin{align*}
{\bb X}^{(n)}& \tood \bb X\quad \text{ and }\quad ({\bb X}^{(n)}-{\bb Y}^{(n)})\toop \bb 0 \quad \text{implies} \quad {\bb Y}^{(n)}\tood \bb X,\\
{\bb X}^{(n)}& \toop \bb X\quad \text{ and }\quad ({\bb X}^{(n)}-{\bb Y}^{(n)})\toop \bb 0 \quad \text{implies} \quad {\bb Y}^{(n)}\toop \bb X,\\
{\bb X}^{(n)}& \tooas \bb X\quad \text{ and }\quad ({\bb X}^{(n)}-{\bb Y}^{(n)})\tooas \bb 0 \quad \text{implies} \quad {\bb Y}^{(n)}\tooas \bb X.
\end{align*}
Proof.
To prove the first statement, it is sufficient to show that for an arbitrary continuous function $h$ that is zero outside a closed and bounded set, $\E(h({\bb Y}^{(n)})) \to \E(h(X))$ (using the portmanteau theorem). Since a continuous function on a closed and bounded set in $\R^d$ (compact) is uniformly continuous, for all $\epsilon>0$ we can find $\delta>0$ such that $\|\bb x-\bb y\| < \delta$ implies $\|h(\bb x )-h(\bb y)\| < \epsilon$. Also, as a continuous function on a closed and bounded set in $\R^d$, $h$ is also bounded, say by $M$, and therefore
\begin{align*}
|\E(h({\bb Y}^{(n)})) - \E(h(X))|
&\leq |\E(h({\bb Y}^{(n)})) - \E(h({\bb X}^{(n)}))|
+ |\E(h({\bb X}^{(n)})) - \E(h(X))|\\
&\leq |\E(h({\bb Y}^{(n)})) - \E(h({\bb X}^{(n)}))| I(\|{\bb X}^{(n)}-{\bb Y}^{(n)}\|\leq \delta) \\
&\quad +
|\E(h({\bb Y}^{(n)})) - \E(h({\bb X}^{(n)}))| I(\|{\bb X}^{(n)}-{\bb Y}^{(n)}\|> \delta)
\\ &\quad + |\E(h({\bb X}^{(n)})) - \E(h(X))|\\
&\leq \epsilon+2M \P(\|{\bb X}^{(n)}-{\bb Y}^{(n)}\|>\delta)
+ |\E(h({\bb X}^{(n)})) - \E(h(X))|.
\end{align*}
The second term in the equation above converges to 0 as $n\to\infty$ since
$({\bb X}^{(n)}-{\bb Y}^{(n)})\toop \bb 0$. The third term also converges to zero since ${\bb X}^{(n)} \tood \bb x$. Since $\epsilon$ was arbitrarily chosen, it follows that $|\E(h({\bb Y}^{(n)})) - \E(h(X))|$, or $\E(h({\bb Y}^{(n)})) \to \E(h(X))$.
To prove the second statement, we note that if $\|{\bb Y}^{(n)}-\bb X\|>\epsilon$ occurs, then by the triangle inequality
$\|{\bb Y}^{(n)}-{\bb X}^{(n)}\|+\|{\bb X}^{(n)}-\bb X\|>\epsilon$ occurs, implying that
\[
\{\|{\bb Y}^{(n)}-\bb X\|>\epsilon\}
\subset \{\|{\bb Y}^{(n)}-{\bb X}^{(n)}\|>\epsilon/2\}
\cup \{\|{\bb X}^{(n)}-\bb X\|>\epsilon/2\}.
\]
This implies that
\begin{align*}
\P(\|{\bb Y}^{(n)}-\bb X\|>\epsilon) \leq
\P(\|{\bb Y}^{(n)}-{\bb X}^{(n)}\|>\epsilon/2) + \P(\|{\bb X}^{(n)}-\bb X\|>\epsilon/2)
\end{align*}
which converges to 0 as $n\to\infty$ since
${\bb X}^{(n)} \toop \bb x$ and $({\bb X}^{(n)}-{\bb Y}^{(n)})\toop \bb 0$.
The third statement follows from arithmetic of deterministic limits, which apply since we have convergence with probability 1.
Corollary 8.11.1.
\begin{align*}
{\bb X}^{(n)}& \tood \bb x\quad \text{ and }\quad {\bb Y}^{(n)}\tood \bb c \quad \text{implies} \quad ({\bb X}^{(n)},{\bb Y}^{(n)})\tood (\bb X,\bb c),\\
{\bb X}^{(n)}& \toop \bb x\quad \text{ and }\quad {\bb Y}^{(n)}\toop \bb c \quad \text{implies} \quad ({\bb X}^{(n)},{\bb Y}^{(n)})\toop (\bb X,\bb c),\\
{\bb X}^{(n)}& \tooas \bb x\quad \text{ and }\quad {\bb Y}^{(n)}\tooas \bb c \quad \text{implies} \quad ({\bb X}^{(n)},{\bb Y}^{(n)})\tooas (\bb X,\bb c).
\end{align*}
Proof.
Since ${\bb Y}^{(n)}\tood \bb c$, we also have ${\bb Y}^{(n)}\toop \bb c$, implying that
\begin{align*}
\P(\| ({\bb X}^{(n)}, {\bb Y}^{(n)})- ({\bb X}^{(n)},\bb c) \| >\epsilon )
= \P(\|{\bb Y}^{(n)}-\bb c\|>\epsilon)\to 0.
\end{align*}
Let $h$ be an arbitrary continuous and bounded function. The statement
$\E(h({\bb X}^{(n)},\bb c))\to \E(h(\bb X,\bb c))$ follows from the fact that ${\bb X}^{(n)}\tood \bb X$ and the portmanteau theorem. Combining this with Slutsky's theorem shows that $({\bb X}^{(n)},{\bb Y}^{(n)})\tood (\bb X,\bb c)$, which proves the first statement.
To prove the second statement, we note that
\begin{multline*}
\P( \|({\bb X}^{(n)}, {\bb Y}^{(n)}) - (\bb X,\bb Y) \| >\epsilon)
\leq \P(\|{\bb X}^{(n)}-\bb X \|>\epsilon/\sqrt{2})\\ +
\P(\|{\bb Y}^{(n)}-\bb Y \|>\epsilon/\sqrt{2}),
\end{multline*}
and that the two terms in the right hand side above converge to 0 as $n\to\infty$.
The third statement follows from arithmetic of deterministic limits, which apply since we have convergence with probability 1.
Corollary 8.11.2.
If $\bb f$ is a continuous function, then
\[{\bb X}^{(n)}\tood \bb X \quad \text{ and }\quad {\bb Y}^{(n)}\tood \bb c\quad \text{imply}\quad \bb f ({\bb X}^{(n)},{\bb Y}^{(n)}) \tood \bb f({\bb X}^{(n)},\bb c).\]
Proof.
This follows from previous corollary and the continuous mapping theorem.
Example 8.11.1.
If $X^{(n)}\tood X$ and $Y^{(n)}\toop c$ for some constant $c$, then
- $X^{(n)} Y^{(n)}\tood Xc$
- $X^{(n)}+Y^{(n)}\tood X+c$
- $X^{(n)}/Y^{(n)}\tood X/c$,
where we assume in the last expression that $c\neq 0$.