Probability

The Analysis of Data, volume 1

Metric Spaces: Basic Definitions

B.1. Basic Definitions

This chapter describes basic metric topology and the Euclidean spaces. It is essential for developing limit theorems, differentiation, and integration.
Definition B.1.1. Let $\mathcal{X}$ be a set. A function $d:\mathcal{X} \times \mathcal{X}\to\mathbb{R}$ is called a distance function if and only if the following properties hold for all $x,y, z\in \mathcal{X}$
  1. non-negativity: $d(x,y)\geq 0$
  2. positivity: $d(x,y)=0$ if and only if $x=y$
  3. symmetry: $d(x,y)=d(y,x)$
  4. triangle inequality: $d(x,z)\leq d(x,y)+d(y,z)$.
The pair $(\mathcal{X},d)$ is called a metric space (we sometimes refer to $\mathcal{X}$ as a metric space if no confusion arises).

The definitions below assume a metric space $(\mathcal{X},d)$. The complement $A^c$ of a set $A\subset \mathcal{X}$ is taken with respect to the metric space: $A^c=\mathcal{X}\setminus A$.

Definition B.1.3. The open ball with center $x\in\mathcal{X}$ and radius $r>0$ is \[B_r(x)=\{y\in\mathcal{X} : d(x,y) < r\}.\]
Definition B.1.3. A set $A\subset\mathcal{X}$ is an open set if it is a union of open balls. A set $A$ is a closed set if it is the complement of an open set. A set $A$ is bounded if $A \subset B_r(x)$ for some $r>0$ and $x$.
Proposition B.1.1. If $G_{\alpha}, \alpha\in Q$ are open sets and $F_{\alpha}, \alpha\in Q$ are closed sets then
  1. $\cup_{\alpha\in Q} G_{\alpha}$ is open
  2. $\cap_{\alpha\in Q} F_{\alpha}$ is closed.
If $G_1,\ldots,G_n$ are open sets and $F_1,\ldots,F_n$ are closed sets then
  1. $\cap_{i=1}^n G_i$ is open
  2. $\cup_{i=1}^n F_i$ is closed.

Note that in the first part of the proposition above there are arbitrarily many open and closed sets, and in the second part there are only a finite number.

Proof. By definition, a union of open sets is a union of open balls. Using De-Morgan's rule (Corollary A.1.1) and is therefore open. The complement of an intersection of closed sets is a union of open sets, which is an open set. This implies that an intersection of closed sets is closed.

To prove the second part, assume that $G_1,\ldots,G_n$ are open sets and pick an arbitrary point $x$ in their intersection. Since open sets are union of open balls, the point $x$ is in $n$ different open balls with different centers and radii. We can find an open ball $B_{\rho_x}(x)$ centered at $x$ that is a subset of the intersection of the $n$ open balls, and is therefore a subset of $\cap_{i=1}^n G_i$. It follows that \[\bigcup_{x\in\cap_{i=1} G_i} B_{\rho_x}(x) =\bigcap_{i=1}^n G_i,\] showing that $\cap_{i=1}^n G_i$ is a union of open balls and therefore an open set. The complementary result follows from De-Morgan's rule, as in the first part of the proof.

In the examples below, we use the metric space $(\R,d)$, where $d(x,y)=|x-y|$. Verifying that $d(x,y)=|x-y|$ is indeed a distance function on $\R$ is straightforward. In Section B.4. we describe this metric space and its high dimensional generalization in more detail.

Example B.1.1. In $\R$, the open interval $(a,b)$ is open (open ball), as is $(a,b)\cup (c,d)$ (union of two open balls). The set $(a,+\infty)$ is a union of infinitely many open balls \[(a,+\infty)=\bigcup_{b:b\geq a+1} B_1(b)\] and is therefore also an open set. Similarly, the set $(-\infty,a)$ is open. The closed interval $[a,b]$ is a closed set since its complement $(-\infty,a)\cup (b,\infty)$ is a union of two open sets and therefore is open. The half open interval $(a,b]$ is neither open nor closed.
Example B.1.2. The set $\R$ is open since it is a union of open balls $\cup_{n\in\mathbb{N}} B_n(0)$, and its complement $\emptyset$ is closed. On the other hand, $\emptyset$ is an intersection of two open sets, for example $\emptyset=(1,2)\cap (3,4)$, which implies that $\emptyset$ is open and its complement $\R$ is closed. We thus have that the sets $\R$ and $\emptyset$ are both open and closed.
Definition B.1.4. A point $x$ is an interior point of a set $A$ if $B_r(x)\subset A$ for some $r>0$. The set of interior points of $A$ is denoted by $\text{int}\, A$. A point $x$ is a boundary point of a set $A$ if for every $r>0$, $B_r(x)$ contains at least one point in $A$ and one point not in $A$. The set of boundary points of $A$ is denoted by $\partial A$. The closure of a set $A$, denoted by $\overline{A}$, is the union of its interior and its boundary: \[ \overline{A}=(\text{int}\, A)\cup (\partial A).\]
Proposition B.1.1. Let $A, F, G$ be an arbitrary set, a closed set, and an open set, respectively, in a metric space $(\mathcal{X},d)$. Then \begin{align*} \text{int}\, G &= G.\\ \text{int}\, A& \subset A \subset \overline{A}\\ F&=\overline{F}. \end{align*}
Proof. We start by proving the first property. Let $x$ be an arbitrary element of $G$. Since $G$ is a union of open balls, there exists an open ball containing $x$. There exists an open ball $B_r(x)$ contained in that open ball, implying that $x$ is an interior point. The second property is obvious. To prove the third property note that $x \in F^c$ is contained in an open ball $B_{r'}(y) \subset F^c$ and choosing $r$ small enough we have $B_{r}(x) \subset B_{r'}(y) \subset F^c$. It follows that $x$ is not a boundary point, implying that $\partial F \subset F$ and therefore $\overline{F}\subset F$. The inclusion $F\subset \overline{F}$ is obvious.
Example B.1.3. In $\R$, we have \begin{align*} \text{int}\, [a,b)&=(a,b)\\ \partial [a,b)&=\{a,b\}\\ \overline{[a,b)}&=[a,b]. \end{align*}
Definition B.1.5. A metric space is second countable if there exists a countably infinite collection of open sets $\mathcal{G}=\{G_n,n\in\mathbb{N}\}$ such that every open set can be expressed as a union of elements in $\mathcal{G}$.
Definition B.1.6. A set $A$ is compact if every open covering has a finite sub-covering. In other words, if $\mathcal{U}$ is a set of open sets whose union contains $A$, there exists a finite number of sets in $\mathcal{U}$ that also contain $A$.