Probability
The Analysis of Data, volume 1
Measure Theory: Independent Sigma-Algebras
$
\def\P{\mathsf{\sf P}}
\def\E{\mathsf{\sf E}}
\def\Var{\mathsf{\sf Var}}
\def\Cov{\mathsf{\sf Cov}}
\def\std{\mathsf{\sf std}}
\def\Cor{\mathsf{\sf Cor}}
\def\R{\mathbb{R}}
\def\c{\,|\,}
\def\bb{\boldsymbol}
\def\diag{\mathsf{\sf diag}}
\def\defeq{\stackrel{\tiny\text{def}}{=}}
$
E.4. Independent $\sigma$-Algebras*
Definition E.4.1.
Let $(\Omega,\mathcal{F},\mu)$ be a measure space. Two events $A,B\in\mathcal{F}$ are independent if $\mu(A\cap B)=\mu(A)\mu(B)$. A finite number of events $A_1,\ldots,A_n\in\mathcal{F}$ are independent if \[ \mu(A_1\cap\cdots A_n)=\mu(A_1)\cdots \mu(A_n).\]
An infinite collection of events $A_{\theta}, \theta\in\Theta$ is independent if for every $k\in\mathbb{N}$ and for every finite subset of distinct events $A_{\theta_1},\ldots,A_{\theta_k}$, $\theta_1,\ldots,\theta_k\in\Theta$ we have
\[\mu(A_{\theta_1}\cap\ldots\cap A_{\theta_k})=\mu(A_{\theta_1})\cdots \mu(A_{\theta_k}).\]
The definitions above of set independence extends to independence of sets of sets.
Definition E.4.2.
Let $(\Omega,\mathcal{F},\mu)$ be a measure space. A finite collection $\mathcal{A}_1,\ldots,\mathcal{A}_n$ of subsets of $\mathcal{F}$ are independent if every selection of sets from these sets $A_1\in\mathcal{A}_1,\ldots,A_n\in\mathcal{A}_n$ are independent. An infinite collection $\mathcal{A}_{\theta}, \theta\in\Theta$ of subsets of $\mathcal{F}$ are independent if every finite number of elements $\theta_1,\ldots\theta_k\in\Theta$, the sets $\mathcal{A}_{\theta_1},\ldots,\mathcal{A}_{\theta_k}$ are independent.
Proposition E.4.1.
Independence of a collection of $\pi$-systems $\mathcal{A}_{\theta}, \theta\in\Theta$ implies independence of the generated $\sigma$-algebras $\sigma(\mathcal{A}_{\theta}), \theta\in\Theta$.
Proof.
We prove the result in the case that $\Theta$ is a finite set of size $n$, which we denote by $\Theta=\{1,2,\ldots,n\}$. An extension for infinite $\Theta$ is available in (Billingsley, 1995)
We assume independence of the $\pi$-systems $\mathcal{A}_1,\ldots,\mathcal{A}_n$. First, note that $\mathcal{A}_i'=\mathcal{A}_i\cup \{\Omega\}$ are also $\pi$-systems. Fix $A_i'\in\mathcal{A}_i'$ for $i=2,\ldots,n$ and consider the set $\mathcal{L}$ of sets $L$ for which
\[ \P(L\cap A_2'\cap\cdots\cap A_n')=\P(L)\P(A_2')\cdots \P (A_n').\]
We show next that $\mathcal{L}$ is a $\lambda$-system. The first condition holds since
\begin{align*}
\P(\Omega \cap A_2'\cap\cdots\cap A_n') &= 1\cdot \P(A_2')\cdots \P(A_n') =
\P(\Omega)\cdot \P(A_2')\cdots \P(A_n').
\end{align*}
The second condition holds since $U\in\mathcal{L}$ implies
\begin{align*}
\P(U^c \cap A_2'\cap\cdots\cap A_n') &= \P((A_2'\cap\cdots\cap A_n')\setminus U)
\\ &= \P(A_2'\cap\cdots\cap A_n') - \P(A_2'\cap\cdots\cap A_n'\cap U)
\\ &= \P(A_2')\cdots \P(A_n') - \P(A_2')\cdots \P(A_n') \P(U)\\
&= \P(A_2')\cdots \P(A_n') \P(U^c).
\end{align*}
The third property follows from the fact that for a disjoint sequence of sets $U_n\in\mathcal{L}, n\in\mathbb{N}$ implies
\begin{align*}
\P((\cup_n U_n) \cap A_2'\cap\cdots\cap A_n')
&= \P(\cup_n (U_n\cap A_2'\cap\cdots\cap A_n')) \\ &= \sum_n \P(U_n\cap A_2'\cap\cdots\cap A_n' ) \\ &= \sum_n \P(U_n)\P(A_2')\cdots \P(A_n') \\ &= \P(A_2')\cdots \P(A_n')\P(\cup_n U_n).
\end{align*}
Since $\mathcal{L}$ is a $\lambda$-system that also contains the $\pi$-system $\mathcal{A}_1'$, by Dynkin's theorem (Proposition
E.3.4) we have that
$\sigma(\mathcal{A}_1'), \{A_2'\}, \ldots, \{A_n'\}$ are independent. Since $A_i'$ were selected arbitrarily from $\mathcal{A}_i'$, it follows that $\sigma(\mathcal{A}_1'), \mathcal{A}_i, \ldots, \mathcal{A}_i$ are independent as well. Repeating this argument multiple times we have that $\sigma(\mathcal{A}_1'), \sigma(\mathcal{A}_2'), \ldots, \sigma(\mathcal{A}_n)$ are independent.
Corollary E.4.1.
Let $(\Omega,\mathcal{F},\P)$ be a probability measure space and $A_{ij}$ be a potentially infinite array of independent events (here $i,j$ vary over a finite or countably infinite set). Then the $\sigma$-algebras generated by the rows of the array
$\mathcal{F}_i=\sigma(\{A_{ij}: \forall j\})$ are independent.
Proof.
We define the sets $\mathcal{C}_i$ of all finite intersections of sets in the $i$-row of the array and consider arbitrary sets $C_i\in\mathcal{C}_i$. Then for a finite set $I\subset \mathbb{N}$ of indices, we have
\begin{align*}
\P\left(\bigcap_{i\in I} C_i\right) &= \P\left(\bigcap_{i\in I} \bigcup_j A_{ij}\right)
= \prod_{i\in I}\prod_j \P(A_{ij}) = \prod_{i\in I} \P(C_i).
\end{align*}
Note that above both $i$ and $j$ vary over a finite set, justifying the equality signs. This implies that the sets $\mathcal{C}_i$, for all $i$, are independent. Using the proposition above we have independence of $\sigma(\mathcal{C}_i)=\mathcal{F}_i$.