Probability

The Analysis of Data, volume 1

Random Vectors: Random Vectors and Independent sigma-Algebras*

4.9. Random Vectors and Independent $\sigma$-Algebras*

As mentioned in Section 2.5, a random variable is a measurable function from the measurable space $(\Omega,\mathcal{F})$ to the measurable space $(\R,\mathcal{B}(\R))$. Similarly, a random vector is a measurable function from $(\Omega,\mathcal{F})$ to $(\R^d,\mathcal{B}(\R^d))$. This ensures that probabilities $\P(\bb X\in A)$ are defined for all $A\in\mathcal{B}(\R^d)$.

Definition 4.9.1. The sets $\mathcal{A}_1, \ldots, \mathcal{A}_n$ of events are independent if $A_1 \in \mathcal{A}_1, \ldots, A_n \in \mathcal{A}_n$ implies that $A_1, \ldots, A_n$ are independent (see Definition 1.5.2).
Proposition 4.9.1. If $\mathcal{A}_1,\ldots,\mathcal{A}_n$ are independent $\pi$-systems, then $\sigma(\mathcal{A}_1),\ldots,\sigma(\mathcal{A}_n)$ are independent.

Both the definition and proposition above extend from a finite $n$ to a potentially infinite number of sets.

Proof. We denote the probability measure space by $(\Omega,\mathcal{F},\P)$, note that $\mathcal{A}_i'=\mathcal{A}\cup\{\Omega\}$, $i=1,\ldots,n$ are $\pi$-systems, and denote $\mathcal{L}\subset\mathcal{F}$ to be the set of all sets $B$ for which \begin{align*} \P(B \cap A_2\cdots\cap A_n) = \P(B)\P(A_2)\cdots \P(A_n) \end{align*} for some fixed sets $A_2\in\mathcal{A}_2, \ldots, A_n\in\mathcal{A}_n$.

We verify below that $\mathcal{L}$ is a $\lambda$-system. We have $\Omega\in\mathcal{L}$ since \begin{align*} \P(\Omega\cap A_2\cap\cdots A_n)&=\P(A_2\cap\cdots\cap A_n)=\P(A_2)\cdots \P(A_n). \end{align*} The set $\mathcal{L}$ is closed under complement since $B\in\mathcal{L}$ implies that \begin{align*} \P(B^c\cap A_2\cap\cdots A_n) &=\P((\Omega\setminus B)\cap A_2\cap\cdots A_n)\\ &=\P(\Omega\cap A_2\cap\cdots\cap A_n \setminus B\cap A_2\cdots\cap A_n) \\ &= \P(\Omega\cap A_2\cap\cdots \cap A_n)-\P(B\cap A_2\cdots\cap A_n)\\ &= \P(A_2)\cdots \P(A_n)-\P(B)\P(A_2)\cdots \P(A_n)\\ &=(1-\P(B))\P(A_2)\cdots \P(A_n). \end{align*} The set $\mathcal{L}$ is closed under disjoint union: if $B_k\in\mathcal{L}, k\in\mathbb{N}$ are pairwise disjoint sets whose union is $B$, then \begin{align*} \P((\cup_{k\in\mathbb{N}} B_k )\cap A_2\cap\cdots\cap A_n) &=\P(\cup_{k\in\mathbb{N}} (B_k\cap A_2\cap\cdots\cap A_n)) \\ &=\sum_{k\in\mathbb{N}} \P(B_k\cap A_2\cap\cdots\cap A_n)\\ &= \sum_{k\in\mathbb{N}} \P(B_k)\P(A_2)\cdots \P(A_n)\\ &= \P(A_2)\cdots \P(A_n)\sum_{k\in\mathbb{N}} \P(B_i). \end{align*}

Since $\mathcal{L}$ is a $\lambda$-system that contains the $\pi$-system $\mathcal{A}_1'$, it follows from Dynkin's theorem (Proposition E.3.3) that $\sigma(\mathcal{A}_1),\mathcal{A}_2,\ldots,\mathcal{A}_n$ are independent. Repeating the argument multiple times concludes the proof.