Probability
The Analysis of Data, volume 1
Metric Spaces: Continuity
$
\def\P{\mathsf{P}}
\def\R{\mathbb{R}}
\def\defeq{\stackrel{\tiny\text{def}}{=}}
\def\c{\,|\,}
$
B.3. Continuity
Definition B.3.1.
A function from one metric space to another, $f:A\to B$, is continuous at $p$ if for all $\epsilon>0$ there exists $\delta > 0$ such that
\[ d( x, p) < \delta \qquad \text{implies} \qquad d(f( x),f( p)) < \epsilon.\]
If $f$ is continuous at all $p\in A$ then we say that $f$ is continuous on $A$ or simply continuous.
Note that there are two distance functions in the definition above: one on the metric space $A$, and the other on the metric space $B$. For simplicity, we denote them both using $d$, but nevertheless they may correspond to different functions.
Proposition B.3.1.
Let $f:A\to B$ be a function and $p$ be a limit point of $A$ (there exists a sequence of points in $A$ converging to $ p$). Then
\[ f \text{ is continuous at } p \qquad \text{if and only if}
\qquad \lim_{ x\to p}f( x)=f( p).\]
Proof.
The proof follows from comparing the definitions of a continuous function and the limit of a function.
Proposition B.3.2 (Continuity Preserves Limits).
Let $f:A\to B$ be a continuous function and $x^{(n)}, n\in\mathbb{N}$ be a convergent sequence. Then
\[ \lim_{n\to\infty} f({ x}^{(n)}) = f\left(\lim_{n\to\infty} { x}^{(n)}\right). \]
Proof.
The proof follows from the previous proposition.
Proposition B.3.3.
If $f:A\to B$ and $g:B\to C$ are continuous functions then their composition $g\circ f:A\to C$ is also a continuous function.
Proof.
Since $g$ is continuous, for $\epsilon > 0$ there exists $\alpha$ such that
\[ d( u, w) < \alpha' \quad\Rightarrow\quad d(g( u),g( w)) < \epsilon.\]
Since $f$ is continuous, for $\alpha > 0$ there exists $\delta > 0$ such that \[ d( x, p) < \delta\quad \Rightarrow \quad d(f( x),f( p)) < \alpha.\]
Combining the two results above with $\alpha=\alpha'$, we see that for all $\epsilon > 0$ there exists $\delta > 0$ such that
\[ d( x, p) < \delta \quad \Rightarrow\quad d(g(f( x)),g(f( p))) < \epsilon.\]
Proposition B.3.4.
A function $f:A\to B$ is continuous if and only if
\[ G \text{ is open } \qquad \text{implies} \qquad f^{-1}(G) \text{ is open}.\]
Proof.
We assume that $f$ is continuous and consider an open set $G\subset B$ and a point $x\in A$ such that $f(x)\in G$. Since $G$ is open, every point in $G$ is an interior point (see previous subsection). Therefore, there $\epsilon>0$ such that $B_{\epsilon}(f(x))\subset G$. Since $f$ is continuous at $x$, there exists $\delta > 0$ such that $d(y,x) < \delta$ implies $d(f(y),f(x)) < \epsilon$. It follows that $x$ is lies in an open ball that is contained in $f^{-1}(G)$. Repeating this argument for all $x$ such that $f(x)\in G$, we get that $f^{-1}(G)$ is a union of open balls, and therefore $f^{-1}(G)$ is open.
Conversely, we assume that $f^{-1}(G)$ is open whenever $G$ is open and consider an arbitrary $x\in A$. The set $B_{\epsilon}(f(x))$ is an open ball, and therefore
$f^{-1}(B_{\epsilon}(f(x)))$ is open as well. This implies that $f^{-1}(B_{\epsilon}(f(x)))$ contains an open ball $B_{\delta}(x)$ for which $f(B_{\delta}(x))\subset B_{\epsilon}(f(x))$. The continuity of the function $f$ follows.
The above proposition is sometimes used to define continuous functions. This leads to a more general definition of continuous functions $f:A\to B$ between sets that are not necessarily metric spaces.
Definition B.3.2.
A function $f:A\to B$ between two metric spaces is uniformly continuous on $A$ if for all $\epsilon > 0$ there exists $\delta$ such that
\[ d(x,y) < \delta \qquad \text{implies}\qquad d(f(x),f(y)) < \epsilon.\]
The difference between a function that is continuous on $A$ and uniformly continuous on $A$ is that, in the former case, $\delta$ may vary depending on $x$ in addition to $\epsilon$, and in the latter case, there is a single $\delta$ for each $\epsilon$.
The following two propositions connect compactness and continuity. Proofs are available in most real analysis textbook, for example (Rudin, 1976).
Proposition B.3.5.
Let $f:A\to B$ be a continuous function between two metric spaces. If $A$ is compact, then $f$ is uniformly continuous on $A$.
Proposition B.3.6.
A continuous function $f:A\to\R$ on a compact set $A$ attains $\inf_{x\in A} f(x)$, $\sup_{x\in A} f(x)$ and all values in between. In this case we refer to the supremum and infimum as the maximum and minimum of $f$ over $A$, and denote them $\max_{x\in A}f(x)$ and $\min_{x\in A}f(x)$, respectively.