Probability

The Analysis of Data, volume 1

Measure Theory: Important Measure Functions

E.5. Important Measure Functions*

We discuss three important measure functions: the discrete measure, the Lebesgue measure, and the Lebesgue-Stieltjes measure.

E.5.1 Discrete Measure Functions*

In this section, we consider a general method to define measures on a finite $\Omega$ or a countably infinite $\Omega$. In this case, we use the $\sigma$-algebra $2^{\Omega}$ - the power set of $\Omega$.

We define the discrete measure on the measurable space $(\Omega,2^{\Omega})$ associated with a set of non-negative numbers $\{p_{\omega}: \omega\in\Omega\}$ as \[\mu(A)=\sum_{\omega\in A} p_{\omega}.\] The function $\mu:2^{\Omega} \to \R$ satisfies $\mu(\emptyset)=0$ and is countably additive, implying that it is a measure. If $\sum_{\omega\in\Omega} p_{\omega}=1$, the discrete measure $\mu$ is also a probability measure $\P$.

The Lebesgue Measure*

In the Section E.5.1 we saw our first example of a measure function, defined on a finite or countably infinite $\Omega$. It is considerably harder to define a meaningful measure on $\R$ or on intervals. We describe in this section a measure function $\mu$ on the interval $\Omega=(0,1]$, called the Lebesgue measure, that agrees with our intuitive notion of length. Specifically, if $A$ is a disjoint union of intervals, $\mu(A)$ is the sum of the lengths of these intervals. The Lebesgue measure may also be defined for other intervals in $\R$, for the entire real line $\R$, and even for multidimensional Euclidean spaces $\R^d$. It is also the basis for the Lebesgue-Stieltjes measure that serves a fundamental role in probability theory.

We start by defining the $\sigma$-algebra that will be used to define the Lebesgue measure.

Definition E.5.1. The Borel $\sigma$-algebra associated with a metric space $(\Omega,d)$ is $\sigma(\mathcal{C})$ where $\mathcal{C}$ is the set of all open sets in the metric space $(\Omega,d)$. We denote this $\sigma$-algebra by $\mathcal{B}(\Omega)$ or simply $\mathcal{B}$ when no confusion arises.
Proposition E.5.1. Let $\mathcal{A}$ be the set of all open balls in $\R^d$. Then $\sigma(\mathcal{A})=\mathcal{B}(\R^d)$.
Proof. Since all open balls are open sets we have $\sigma(\mathcal{A})\subset \mathcal{B}(\R^d)$. Since $\R^d$ is second countable (Proposition B.4.5) every open set can be expressed as a countable union of open balls in $\mathcal{A}$, implying that all open sets are also in $\sigma(\mathcal{A})$ (since $\sigma(\mathcal{A})$ is closed under countable unions) and consequentially $\mathcal{B}(\R^d)\subset \sigma(\mathcal{A})$.
Proposition E.5.2. \[\mathcal{B}(\R) = \sigma\left(\left\{(-\infty,x]: x\in\R\right\}\right).\]
Proof. The previous proposition showed that the Borel sets in $\R$ are generated by the set of open intervals $(a,b)$. It is thus sufficient to show (i) $(a,b)\in \sigma(\{(-\infty,x]: x\in\R\})$, and (ii) $(-\infty,x]\in\mathcal{B}(\R)$. These two assertions follow from the equations below. \begin{align*} (a,b) &= \left(\bigcup_{n\in\mathbb{N}} (-\infty,b-1/n]\right) \cap (-\infty,a]^c\\ (-\infty,x] &= \bigcup_{n\in\mathbb{N}} (x,x+n)^c. \end{align*}
Proposition E.5.3. Let $\mathcal{A}$ be the set of all half-open intervals in $(\alpha,\beta]$ of the form $(a,b]$, ($\alpha\leq a < b\leq \beta$) and $\mathcal{D}$ the set of all unions of a finite number of disjoint half open intervals from $\mathcal{A}$. Then $\mathcal{D}$ is an algebra in $\Omega=(\alpha,\beta]$, and $\sigma(\mathcal{D})=\mathcal{B}((\alpha,\beta])$.
Proof. With no loss of generality, we assume in the proof that $\alpha=0$ and $\beta=1$. We first show that $\mathcal{D}$ is an algebra over $\Omega=(0,1]$. It is obvious that $(0,1]\in\mathcal{A}$ and therefore $\Omega\in\mathcal{D}$. The complement of a disjoint union of half open intervals $(a,b]$ is a disjoint union of half open intervals. Similarly, a finite union of disjoint unions of half open intervals is a disjoint union of half open intervals. It thus follows that $\mathcal{D}$ is an algebra.

Proposition E.5.1 and Proposition E.1.5 imply that $\mathcal{B}((0,1])$ is generated by the open balls in $(0,1]$, or intervals $(a,b)$ with $0\leq a < b\leq 1$. To show that $\mathcal{B}((0,1])=\sigma(\mathcal{D})$ it suffices to show that (i) $(a,b)\in \sigma(\mathcal{D})$, and that (ii) $(a,b]\in \mathcal{B}((0,1])$. The first claim holds since $(a,b)=\cup_{n\in\mathbb{N}} (a,b-1/n]$ and the second holds since $(a,b]=\cap_{n\in\mathbb{N}} (a,b+1/n) = (\cup_{n\in\mathbb{N}} (a,b+1/n)^c)^c$.

Proposition E.5.4. Using the notations from the previous proposition, we define the Lebesgue measure $\mu$ on $\Omega=(\alpha,\beta]$ and the algebra $\mathcal{D}$ of half open intervals (see the proposition above) as follows: \[\mu(A)=\sum_{i=1}^k |b_i-a_i|\] where $A$ is represented by the disjoint union of intervals $A=\cup_{i=1}^k (a_i,b_i]$. The measure has a unique extension to the Borel $\sigma$-algebra $\mathcal{B}((\alpha,\beta])$.
Proof. With no loss of generality, we assume in the proof that $\alpha=0$ and $\beta=1$. The function $\mu$ is a measure since it assigns value 0 to the empty set, it is non-negative, and given a sequence of disjoint sets $D_n\in \mathcal{D},n\in\mathbb{N}$ we have $\mu(\cup D_n)=\sum \mu(D_n)$.

Since $\mathcal{D}$ is an algebra and $\sigma(\mathcal{D})=\mathcal{B}(0,1])$ (by the proposition above), we can use Caratheodory's extension theorem (Proposition E.3.10) to assert that there is a unique extension of $\mu$ to the Borel $\sigma$-algebra $\mathcal{B}((\alpha,\beta])$.

We make the following observations.

  1. The Lebesgue measure may be defined on different types of intervals $(\alpha,\beta)$, $[\alpha,\beta]$, or $[\alpha,\beta)$ in accordance with the definition above on intervals of the form $[\alpha,\beta)$. Consistency of the Lebesgue measure over all of these intervals follows from the fact that the Lebesgue measure (length) of a single point is zero.
  2. When $\beta-\alpha=1$, the Lebesgue measure is also a probability measure. In other cases, we can define a probability measure associated with the Lebesgue measure on $(\alpha,\beta]$ as follows \[ \P(A) = \frac{1}{\beta-\alpha}\, \mu(A).\] This probability function is the classical probability function on continuous spaces, defined in Section 1.4.
  3. Letting $\alpha\to-\infty$ and $\beta\to+\infty$ generalizes the Lebesgue measure to $\Omega=\R$ and the $\sigma$-algebra $\mathcal{B}(\R)$. Note that in this case $\mu(\Omega)=+\infty$, implying that the Lebesgue measure on $\R$ is not a probability measure, and cannot be converted to one using the normalization that is described above.
  4. The Lebesgue measure can be generalized to subsets of higher dimensional Euclidean spaces $\R^d$. The resulting measure generalizes area and volume for the cases $d=2$ and $d=3$, respectively. These generalizations will be developed in Section F.5.

E.5.3 The Lebesgue-Stieltjes Measure*

The Lebesgue-Stieltjes measure is a useful generalization of the Lebesgue measure, where some regions of $\R$ have higher measure than others. The relation between measures across different regions of $\R$ is specified using a non-decreasing right continuous function.
Definition E.5.3. Let $F:(\alpha,\beta]\to [0,\infty)$ be a non-decreasing right continuous function. Using the definitions from Proposition E.5.3, we define the Lebesgue-Stieltjes measure on $\Omega=(\alpha,\beta]$ and the algebra $\mathcal{D}$ as \[ \mu_F(A) = \sum_{i=1}^k F(b_i)-F(a_i) \] where $A\in\mathcal{D}$ is represented as a finite union of disjoint half open intervals $A=\cup_i (a_i,b_i]$.
Proposition E.5.5. The Lebesgue-Stieltjes measure defined above on $\Omega=(\alpha,\beta]$ and the algebra $\mathcal{D}$ has a unique extension to $\mathcal{B}((\alpha,\beta])$.
Proof. The proof is identical to the proof of Proposition E.5.4.

As in the case of the Lebesgue measure, letting $\alpha\to-\infty$ and $\beta\to\infty$ generalizes the Lebesgue-Stieltjes measure to $\Omega=\R$ and the $\sigma$-algebra $\mathcal{B}(\R)$.