Probability

The Analysis of Data, volume 1

Random Vectors: Marginal Random Vectors

4.3. Marginal Random Vectors

We can relate probabilities involving $\bb{X}\in A$ to probabilities involving the one dimensional components $X_i$ or, more generally, to probabilities involving a subset of the dimensions of $\bb{X}$. The most convenient way to do so is to relate the corresponding pmfs, cdfs, and pdfs.

Proposition 4.3.1. \begin{align*} p_{X_i}(x_i) &= \sum_{x_1} \cdots \sum_{x_{i-1}}\sum_{x_{i+1}} \cdots\sum_{x_n} p_{\bb{X}}(x_1,\ldots,x_n)\\ f_{X_i}(x_i) &= \idotsint f_{\bb{X}}(x_1,\ldots,x_n) \,dx_1\cdots dx_{i-1} dx_{i+1} dx_n \\ F_{X_i}(x_i) &= \lim_{x_1\to\infty}\cdots \lim_{x_i\to\infty}\lim_{x_{i+1}\to\infty}\cdots \lim_{x_n\to\infty} F_{\bb{X}}(x_1,\cdots,x_n). \end{align*}
Proof. The proof follows from the fact that the following two sets are equal \[ \{X_1\in S_1\} = \{(X_1,X_2,\ldots,X_n)\in S_1\times \mathbb{R} \times \cdots\times \mathbb{R}\}.\]
Example 4.3.1. In the two-dimensional case \begin{align*} p_{X_1}(x_1)&=\sum_{x_2} p_{X_1,X_2}(x_1,x_2)\\ f_{X_1}(x_1)&=\int_{-\infty}^{\infty} f_{X_1,X_2}(x_1,x_2) \, dx_2\\ F_{X_1}(x_1) &= \lim_{x_2\to\infty} F_{X_1,X_2}(x_1,x_2). \end{align*}

A result similar to Proposition 4.3.1 (with obvious modifications) applies to higher-order marginals. For example, \begin{align*} F_{X_1,X_4}(x_1,x_4) &= \lim_{x_2\to\infty}\lim_{x_3\to\infty} F_{X_1,X_2,X_3,X_4}(x_1,x_2,x_3,x_4)\\ f_{X_1,X_4}(x_1,x_4) &= \int_{-\infty}^{\infty}\int_{-\infty}^{\infty} f_{X_1,X_2,X_3,X_4}(x_1,x_2,x_3,x_4) \,dx_2 dx_3. \end{align*}

Proposition 4.3.2. The vector $\bb{X}$ is independent if and only if \[F_{\bb{X}}(x_1,\ldots,x_n) = F_{X_1}(x_1)\cdots F_{X_n}(x_n)\] which occurs for discrete random vectors if and only if \[p_{\bb{X}}(x_1,\ldots,x_n) = p_{X_1}(x_1)\cdots p_{X_n}(x_n)\] and for continuous vectors if and only if \[f_{\bb{X}}(x_1,\ldots,x_n) = f_{X_1}(x_1)\cdots f_{X_n}(x_n).\]
Proof*. The proof uses results from measure theory. Formally, the equations in the proposition hold almost everywhere (see Definition F.3.6). If the cdf factorizes then by definition the pdf or the pmf factorize as well. Similarly, if the pmf or the pdf factorize it is easy to see that the cdf factorizes as well. If the components of $\bb X$ are independent, the cdf factorizes.

It remains to show that if the cdf factorizes, the components of the random vector $\bb X$ are independent. From a measure theory perspective, factorization of the cdf is equivalent to independence of \begin{align} \tag{*} \mathcal{A}_k = \{\R^{k-1} \times (-\infty,a] \times \R^{n-k}: a\in\R\}, \qquad k=1,\ldots,n \end{align} under $(\R^n,\mathcal{B}(\R^n),\P')$. Similarly, independence of the components of $\bb X$ corresponds to independence of the sets \begin{align}\tag{**} \mathcal{C}_k = \{\R^{k-1} \times B \times \R^{n-k}: B\in\mathcal{B}(\R) \}, \qquad i=1,\ldots,n \end{align} under $(\R^n,\mathcal{B}(\R^n),\P')$. See Definition 4.9.1 at the end of this chapter for a definition of the concept of independence of sets.

The equivalence between (*) and (**) is asserted by Proposition 4.9.1, which states that the independence of $\mathcal{A}_1,\ldots,\mathcal{A}_n$ implies the independence of $\sigma(\mathcal{A}_1)=\mathcal{C}_1,\ldots,\sigma(\mathcal{A}_n)=\mathcal{C}_n$.

Example 4.3.2. Consider the following pdf \[f_{X,Y}(x,y)=\frac{1}{2\pi}e^{-(x^2+y^2)/2},\] which factorizes as $f_X(x)f_Y(y)$ where $f_X$ and $f_Y$ are the pdfs of $N(0,1)$ Gaussian RVs. This implies that $X,Y$ are independent $N(0,1)$ Gaussian RVs. The factorization leads to simplified calculation of probabilities, for example \begin{align*} \P(3 < X < 6, Y < 9) &= \int_3^6\int_{-\infty}^9 f_{X,Y}(x,y)\,dx dy = \int_3^6\int_{-\infty}^9 f_{X}(x)f_Y(y)\,dx dy \\ &= \int_{3}^6 f_X(x)dx \int_{\infty}^9 f_Y(y)dy. \end{align*}
Example 4.3.4. For a random vector $(X,Y)$ where $X,Y$ are independent exponential random variables with parameters $\lambda_1,\lambda_2$ we have \[f_{X,Y}(x,y)=f_X(x)f_Y(y)=\lambda_1\lambda_2 e^{-\lambda_1 x}e^{-\lambda_2 y}=\lambda_1\lambda_2 e^{-\lambda_1 x-\lambda_2 y}.\] The cdf is \begin{align*} F_{X,Y}(x,y) &= \int_{-\infty}^x\int_{-\infty}^y \lambda_1\lambda_2 e^{-\lambda_1 x-\lambda_2 y} dxdy=\int_{-\infty}^x \lambda_1 e^{-\lambda_1 x} \int_{-\infty}^y \lambda_2 e^{-\lambda_2 y}\\ &= (1- e^{-\lambda_1 x})(1- e^{-\lambda_1 x})=F_X(x)F_Y(y). \end{align*}