The Analysis of Data, volume 1

Measure Theory: Independent Sigma-Algebras

E.4. Independent $\sigma$-Algebras*

Definition E.4.1. Let $(\Omega,\mathcal{F},\mu)$ be a measure space. Two events $A,B\in\mathcal{F}$ are independent if $\mu(A\cap B)=\mu(A)\mu(B)$. A finite number of events $A_1,\ldots,A_n\in\mathcal{F}$ are independent if \[ \mu(A_1\cap\cdots A_n)=\mu(A_1)\cdots \mu(A_n).\] An infinite collection of events $A_{\theta}, \theta\in\Theta$ is independent if for every $k\in\mathbb{N}$ and for every finite subset of distinct events $A_{\theta_1},\ldots,A_{\theta_k}$, $\theta_1,\ldots,\theta_k\in\Theta$ we have \[\mu(A_{\theta_1}\cap\ldots\cap A_{\theta_k})=\mu(A_{\theta_1})\cdots \mu(A_{\theta_k}).\]

The definitions above of set independence extends to independence of sets of sets.

Definition E.4.2. Let $(\Omega,\mathcal{F},\mu)$ be a measure space. A finite collection $\mathcal{A}_1,\ldots,\mathcal{A}_n$ of subsets of $\mathcal{F}$ are independent if every selection of sets from these sets $A_1\in\mathcal{A}_1,\ldots,A_n\in\mathcal{A}_n$ are independent. An infinite collection $\mathcal{A}_{\theta}, \theta\in\Theta$ of subsets of $\mathcal{F}$ are independent if every finite number of elements $\theta_1,\ldots\theta_k\in\Theta$, the sets $\mathcal{A}_{\theta_1},\ldots,\mathcal{A}_{\theta_k}$ are independent.
Proposition E.4.1. Independence of a collection of $\pi$-systems $\mathcal{A}_{\theta}, \theta\in\Theta$ implies independence of the generated $\sigma$-algebras $\sigma(\mathcal{A}_{\theta}), \theta\in\Theta$.
Proof. We prove the result in the case that $\Theta$ is a finite set of size $n$, which we denote by $\Theta=\{1,2,\ldots,n\}$. An extension for infinite $\Theta$ is available in (Billingsley, 1995)

We assume independence of the $\pi$-systems $\mathcal{A}_1,\ldots,\mathcal{A}_n$. First, note that $\mathcal{A}_i'=\mathcal{A}_i\cup \{\Omega\}$ are also $\pi$-systems. Fix $A_i'\in\mathcal{A}_i'$ for $i=2,\ldots,n$ and consider the set $\mathcal{L}$ of sets $L$ for which \[ \P(L\cap A_2'\cap\cdots\cap A_n')=\P(L)\P(A_2')\cdots \P (A_n').\]

We show next that $\mathcal{L}$ is a $\lambda$-system. The first condition holds since \begin{align*} \P(\Omega \cap A_2'\cap\cdots\cap A_n') &= 1\cdot \P(A_2')\cdots \P(A_n') = \P(\Omega)\cdot \P(A_2')\cdots \P(A_n'). \end{align*} The second condition holds since $U\in\mathcal{L}$ implies \begin{align*} \P(U^c \cap A_2'\cap\cdots\cap A_n') &= \P((A_2'\cap\cdots\cap A_n')\setminus U) \\ &= \P(A_2'\cap\cdots\cap A_n') - \P(A_2'\cap\cdots\cap A_n'\cap U) \\ &= \P(A_2')\cdots \P(A_n') - \P(A_2')\cdots \P(A_n') \P(U)\\ &= \P(A_2')\cdots \P(A_n') \P(U^c). \end{align*} The third property follows from the fact that for a disjoint sequence of sets $U_n\in\mathcal{L}, n\in\mathbb{N}$ implies \begin{align*} \P((\cup_n U_n) \cap A_2'\cap\cdots\cap A_n') &= \P(\cup_n (U_n\cap A_2'\cap\cdots\cap A_n')) \\ &= \sum_n \P(U_n\cap A_2'\cap\cdots\cap A_n' ) \\ &= \sum_n \P(U_n)\P(A_2')\cdots \P(A_n') \\ &= \P(A_2')\cdots \P(A_n')\P(\cup_n U_n). \end{align*}

Since $\mathcal{L}$ is a $\lambda$-system that also contains the $\pi$-system $\mathcal{A}_1'$, by Dynkin's theorem (Proposition E.3.4) we have that $\sigma(\mathcal{A}_1'), \{A_2'\}, \ldots, \{A_n'\}$ are independent. Since $A_i'$ were selected arbitrarily from $\mathcal{A}_i'$, it follows that $\sigma(\mathcal{A}_1'), \mathcal{A}_i, \ldots, \mathcal{A}_i$ are independent as well. Repeating this argument multiple times we have that $\sigma(\mathcal{A}_1'), \sigma(\mathcal{A}_2'), \ldots, \sigma(\mathcal{A}_n)$ are independent.

Corollary E.4.1. Let $(\Omega,\mathcal{F},\P)$ be a probability measure space and $A_{ij}$ be a potentially infinite array of independent events (here $i,j$ vary over a finite or countably infinite set). Then the $\sigma$-algebras generated by the rows of the array $\mathcal{F}_i=\sigma(\{A_{ij}: \forall j\})$ are independent.
Proof. We define the sets $\mathcal{C}_i$ of all finite intersections of sets in the $i$-row of the array and consider arbitrary sets $C_i\in\mathcal{C}_i$. Then for a finite set $I\subset \mathbb{N}$ of indices, we have \begin{align*} \P\left(\bigcap_{i\in I} C_i\right) &= \P\left(\bigcap_{i\in I} \bigcup_j A_{ij}\right) = \prod_{i\in I}\prod_j \P(A_{ij}) = \prod_{i\in I} \P(C_i). \end{align*} Note that above both $i$ and $j$ vary over a finite set, justifying the equality signs. This implies that the sets $\mathcal{C}_i$, for all $i$, are independent. Using the proposition above we have independence of $\sigma(\mathcal{C}_i)=\mathcal{F}_i$.