Probability

The Analysis of Data, volume 1

Product Measures

F.4. Product Measures*

Definition F.4.1. Let $(\Omega_t,\mathcal{F}_t)$, $t\in T$ be measurable spaces and consider the Cartesian product set (Definitions A.1.10, A.3.4) $\times_{t\in T} \Omega_t$. A set $A\subset \times_{t\in T} \Omega_t$ is a measurable rectangle if $A=\times_{t\in T} A_t$ for some $A_t\in\mathcal{F}_t$, for all $t\in T$. A set $A\subset \times_{t\in T} \Omega_t$ is a measurable cylinder if $A=A_s \times (\times_{t: t\neq s} \Omega_t)$, $A_s\in\mathcal{F}_s$, $s\in T$. If $A_s$ above is open the set is called an open measurable cylinder and if $A_s=(a,b)$ the set is called a simple open measurable cylinder.

Measurable rectangles are products of sets in the corresponding $\sigma$-algebras $\mathcal{F}_t$, $t\in T$. Some examples of measurable rectangles in $\Omega=\R^3=\R\times \R\times \R$ (assuming the the Borel $\sigma$-algebra on each copy of $\R$) are \begin{align*} &(1,2)\times \R \times [2,3)\\ &([1,2]\cup [3,4]) \times[2,3]\times [4,5]\\ &\R\times \R \times \cup_{n\in\mathbb{N}} [n,n+1/2). \end{align*} For example, the set $\{\bb x: \sum_i x_i=1\}$ is not a measurable rectangle. Figure 4.2 illustrates some sets in $\R^2$ that are measurable rectangles and some sets that are not.

Measurable cylinders $\{A_s \times (\times_{t\neq s} \Omega_t):A_s\in\mathcal{F}_s, s\in T\}$ are special cases of measurable rectangles, in which only one of the products is allowed to differ from $\Omega_t$. A general characterization of the measurable cylinders in $\R^3$ is \begin{align*} &A \times \R \times \R, \qquad A\in\mathcal{B}(\R)\\ &\R \times A \times \R, \qquad A\in\mathcal{B}(\R)\\ &\R \times \R \times A, \qquad A\in\mathcal{B}(\R). \end{align*}

The following definitions constructs a new measurable space from several measurable spaces. The new measurable space, called the product measurable space, leads to the construction of the product measure, which is instrumental in studying probability over Euclidean spaces $\R^n$.

Definition F.4.2. Let $(\Omega_t,\mathcal{F}_t)$, $t\in T$ be measurable spaces. The product measurable space is the pair $(\Omega,\mathcal{F})$ of a set $\Omega$ and a $\sigma$-algebra defined by \begin{align*} \Omega &= \times_{t\in T}\Omega_t \qquad \text{(see Definitions A.1.10, A.3.4)}\\ \mathcal{F} &= \otimes_{t\in T} \mathcal{F}_t \defeq \sigma\left( \left\{ A: A \text{ is a measurable cylinder in }\Omega\right\} \right). \end{align*}

Since $\sigma($all open measurable cylinders) includes all measurable cylinders and therefore \begin{align*} \otimes_{t\in T} \mathcal{F}_t = \sigma\left( \left\{ A: A \text{ is an open measurable cylinder}\right\} \right). \end{align*}

Proposition F.4.1. Recalling Definitions A.1.10, A.3.4, E.5.1, we have \begin{align*} \mathcal{B}(\R\times\R\times\cdots) &= \mathcal{B}(\R) \otimes \mathcal{B}(\R) \otimes \cdots \\ \mathcal{B}(\R\times\cdots\times \R) &= \mathcal{B}(\R) \otimes \cdots\otimes \mathcal{B}(\R). \end{align*} In other words, the $\sigma$-algebra $\mathcal{B}(\R) \otimes \cdots \otimes \mathcal{B}(\R)$ is generated by open sets in $\R^d$ (and similarly for infinite products).

In the proposition above and elsewhere, whenever we refer to the set $\R^{\infty}$, we assume the metric structure defined in Example B.4.4.

Proof. The statement above may be written as $\sigma(\mathcal{L}_1)=\sigma(\mathcal{L}_2)$ where $\mathcal{L}_1$ is the set of open sets in $\R^d$ or $R^{\infty}$ and $\mathcal{L}_2$ is the set of open measurable cylinders. It is sufficient to show that $\mathcal{L}_1\subset \sigma(\mathcal{L}_2)$ and $\mathcal{L}_2\subset \sigma(\mathcal{L}_1)$.

A set in $\mathcal{L}_2$ is an open measurable cylinder and is therefore an open set in $\R^d$ or in $\R^{\infty}$ (see Example B.4.4).

In the case of a finite number of products, let $A$ be an open set in $\R^d$. Then for each $a\in A$ there exists an open ball $B_r(a')$ such that $a\in B_r(a')\subset A$. For each $a\in A$ and each open ball $B_r(a')$ containing $a$ there exists a rectangle $\times_{i=1}^d (a_i,b_i)$ with rational endpoints such that $a\in \times_{i=1}^d (a_i,b_i) \subset B_r(a')\subset A$ and the union of all such open rectangles equals $A$. Since open rectangles are finite intersections of cylinder sets, and there are a countable number of rectangles with rational end-points, $A$ is a countable union of finite intersections of cylinders, implying that $A\in\sigma(\mathcal{B}(\R) \otimes \cdots \otimes \mathcal{B}(\R))$.

The case of an infinite number of products proceeds similarly. By second countability of $\R^{\infty}$ (see Proposition B.4.5), an open set $A\subset R^{\infty}$ is a countable union of finite intersections of cylinders. It follows that $A=\sigma(\mathcal{L}_2)$.

Corollary F.4.1. \[\mathcal{B}(\R^d) = \sigma( \{(a_1,b_1)\times\cdots\times(a_d,b_d): a_i,b_i\in\mathbb{Q}, i=1,\ldots,d\})\] and similarly, $\mathcal{B}(R^{\infty})=\sigma(\mathcal{A})$ where $\mathcal{A}$ is the set of all simple open measurable cylinders with rational endpoints.
Proof. This follows from the proof of the proposition above.
Proposition F.4.2. A function $f:\Omega\to\R^d$, $f(x)=(f_1(x),\ldots,f_d(x))$ is measurable if $f_i:\Omega\to\R$ is measurable.
Proof. Corollary F.4.1 implies that the $\sigma$-algebra $\mathcal{B}(\R^d)$ is generated by the set of all open rectangles. By Proposition E.6.1 it is then sufficient to show that for all open rectangles $A$, $f^{-1}(A)\in\mathcal{F}$. Because $A$ is a rectangle, $f^{-1}(A)=\cap_{i=1}^d f_i^{-1}((a_i,b_i))$ which is a subset of $\mathcal{F}$ since $f_i$ are measurable and since the $\sigma$-algebra of $\Omega$ is closed under finite intersections.

In the following, we concentrate on product of two spaces. This is done for the sake of simplicity, and the definitions and propositions carry over to a product of a finite number of spaces (for example, by induction).

Proposition F.4.3. Let $(X,\mathcal{X})$ and $(Y,\mathcal{Y})$ be two measurable spaces. If $E\subset \mathcal{X}\otimes\mathcal{Y}$ then \begin{align} \{y:(x,y)\in E\} &\in \mathcal{Y}\\ \{x:(x,y)\in E\} &\in \mathcal{X}. \end{align}
Proof. We prove the first statement. The proof of the second statement is similar. Defining the function $\phi:Y\to X\times Y$, $\phi(y)=(x_0,y)$, we have for each measurable rectangle $A\times B$, \[\phi^{-1}(A\times B)=\begin{cases} B \in \mathcal{Y} & x_0\in A\\ \emptyset \in \mathcal{Y} & x_0\not\in A\end{cases}.\] Combined with Proposition E.6.1 and Definition F.4.2, this implies that $\phi$ is a measurable function. It follows that $\{y:(x,y)\in E\}=\phi^{-1}(E)$ is in $\mathcal{Y}$ for all $E\subset \mathcal{X}\otimes\mathcal{Y}$.
Proposition F.4.4. Let $(X,\mathcal{X})$ and $(Y,\mathcal{Y})$ be two measurable spaces and consider the measurable space $(X\times Y,\mathcal{X}\otimes\mathcal{Y})$. If $f:X\times Y\to \R$ is a measurable function on $X\times Y$, then the functions $g(x)=f(x,y_0)$ and $h(y)=f(x_0,y)$ are measurable on $X$ and $Y$ respectively, for each $x_0 \in X,y_0\in Y$.
Proof. Using the notations from the proof of the previous proposition, $h=f\circ \phi$, and since a composition of measurable functions is measurable (Proposition E.6.2), $h$ is measurable. The proof of the measurability of $g$ is similar.
Proposition F.4.5 (Product Measure Theorem). Let $(X,\mathcal{X}, \P_X)$ and $(Y,\mathcal{Y}, \P_Y)$ be two probability measure spaces. Then there is a unique probability measure $ \P_{X\times Y}$ on the measurable space $(X\times Y,\mathcal{X}\otimes\mathcal{Y})$ such that for all measurable rectangles $A\times B$, \[ \P_{X\times Y}(A\times B)= \P_X(A)\cdot \P_Y(B).\]
Proof. By Proposition F.4.3, for all measurable sets $E$ and for all $x_0,y_0$, the sets $\{y: (x_0,y)\in E\}$ and $\{x: (x,y_0)\in E\}$ are measurable sets in $Y$ and $X$, respectively. Since they are measurable sets we can apply the measures $ \P_Y$ and $ \P_X$ to them, obtaining the real valued functions $f_{E}(x)= \P_Y(\{y: (x,y)\in E\})$ and $g_{E}(y)= \P_X(\{x: (x,y)\in E\})$.

We denote the set of sets $E$ for which $f_{E}$ is measurable as $\mathcal{L}$, and show that it is a $\lambda$-system on $X\times Y$. We have $X\times Y \in\mathcal{L}$ since $f_{E}(x)= \P_Y(\{y: (x,y)\in X\times Y\})= \P_Y(Y)$. If $A\in\mathcal{L}$, then $f_{E^c}(x)$ is measurable and therefore $f_{E}(x)= \P_Y(\{y: (x,y)\in A^c\})= \P_Y(Y)-f_{E^c}(x)$ is measurable as well. If $A_n, n\in\mathbb{N}$ is a sequence of disjoint sets in $\mathcal{L}$ then $f_{\cup A_n}(x)= \P_Y(\{y: (x,y)\in \cup_n A_n\})=\sum_{n\in\mathbb{N}}f_{A_n}(x)$ implying that $\cup A_n$ is in $\mathcal{L}$ as well. We have thus shows that $\mathcal{L}$ is a $\lambda$ system. This derivation also shows that $f_E(x)$ and $g_E(x)$ are measure functions for all $x$.

The function $f_E$ is also measurable for all measurable rectangles $E=A\times B$, since $f_{A\times B}(x)=I_A(x) \P_X(B)$. $\mathcal{L}$ is a $\lambda$-system containing the $\pi$-system of measurable rectangles, which generates the $\sigma$ algebra $\mathcal{X}\otimes\mathcal{Y}$. It follows from Dynkin's Theorem (Proposition E.3.4) that $\mathcal{X}\otimes\mathcal{Y}\subset \mathcal{L}$ implying that $f_E$ is measurable. The proof of measurability of $g_E$ is similar.

Since for all $E\in\mathcal{X}\otimes\mathcal{Y}$, the functions $f_E$ and $g_E$ are measurable, we can define the following functions that assign real values to sets in $\mathcal{X}\otimes\mathcal{Y}$ \begin{align} \P_{X\times Y}^{(1)}(E)&=\int_X \P_X(\{y: (x,y)\in E\})\, d \P_X,\\ \P_{X\times Y}^{(2)}(E)&=\int_Y \P_Y(\{y: (x,y)\in E\})\, d \P_Y. \end{align} For measurable rectangles, $f_{A\times B}(x)=I_A(x) \P_X(B)$ and $g_{A\times B}(x)=I_B(x) \P_Y(A)$, implying that ${\P_{X\times Y}}^{(1)}(A\times B)= \P_{X\times Y}^{(2)}(A\times B)= \P_X(A)\cdot \P_Y(B)$. Since measurable rectangles generate $\mathcal{X}\otimes\mathcal{Y}$, it follows from Corollary E.3.1 that $\P_{X\times Y}^{(1)}$ agrees with $\P_{X\times Y}^{(2)}$ on $\mathcal{X}\otimes\mathcal{Y}$. It also follows that if there is any other measure $\P_{X\times Y}^{(3)}$ which assigns the value $\P_{X\times Y}^{(3)}=(A\times B)=\P_X(A)\cdot \P_Y(B)$ on measurable rectangles, then $\P_{X\times Y}^{(3)}$ agrees with $\P_{X\times Y}^{(1)}$ and $\P_{X\times Y}^{(2)}$ on $\mathcal{X}\otimes\mathcal{Y}$.

We conclude with a few comments.
  1. We usually denote the product measure of $\P_X$ and $\P_Y$ as $\P_X\times \P_Y$.
  2. The product measure corresponds to independence between the random variables $X$ and $Y$. Other probability measures are also available on the product space $(X\times Y,\mathcal{X}\otimes\mathcal{Y})$ (see Chapters 4 and 5).
  3. The product measure theorem can be generalized to measures that are not probability functions. See (Billingsley, 1995) for a more general version.
  4. The product measure theorem can be generalized for products of more than two spaces. The generalization is straightforward since the definitions above are associative: \begin{align*} (X\times Y)\times Z &= X\times (Y\times Z)\\ (\mathcal{X}\otimes \mathcal{Y})\otimes \mathcal{Z} &= \mathcal{X}\otimes (\mathcal{Y}\otimes \mathcal{Z})\\ (\P_X\times \P_Y)\times\P_Z &= \P_X\times( \P_Y\times\P_Z). \end{align*} Further extensions exist for a product of an infinite number of measure spaces (both countably infinite and uncountably infinite).