Exercise 2.21 (Independence of real-valued random variables)

Question

Let $X_1,\dots,X_n$ be real scalar random variables. Show that $X_1,\dots,X_n$ are jointly independent if and only if one has

$$\displaystyle {\bf P}\left( \bigwedge_{i=1}^n (X_i \leq t_i) \right) = \prod_{i=1}^n {\bf P}( X_i \leq t_i )$$

for all $t_1,\dots,t_n \in {\bf R}$.

Elu Thingol · Accepted Answer

To demonstrate the equivalence of both conditions, we show that the implications hold in both directions.
\begin{itemize}
\item[$\implies$] This 	extit{only if} part is obvious. As per Definition 16 (Independence), set $S_i := (-\infty,t_i]$ and obtain
\begin{align*}
{\bf P}\left( \bigwedge_{i=1}^n (X_i \leq t_i) ight) &= {\bf P}\left( \bigwedge_{i=1}^n (X_i \in S_i) ight)\
&= \prod_{i=1}^{n} {\bf P}\left( X_i \in S_i ight)\
&= \prod_{i=1}^{n} {\bf P}\left( X_i \leq t_i ight).
\end{align*}

\item[$\impliedby$] For simplicity, we will first work out the case when $n=2$ and resume by induction next; thus, assume that
$$\forall t_1, t_2 \in \mathbf{R}: \quad {\bf P}(X \leq t_1, Y \leq t_2) = {\bf P}(X \leq t_1) \cdot {\bf P}(Y \leq t_2).$$
Our goal is to verify that 
$$\forall S_1, S_2 \in \mathcal{B}: \quad {\bf P}(X \in S_1, Y \in S_2) = {\bf P}(X \in S_1) \cdot {\bf P}(Y \in S_2).$$
After much thinking, one can see that the following lemmas come quite handy when analyzing this problem.

\begin{lemma} (Inverse image of a generator set is a generator set of the inverse image)
Let $(\Omega, \mathcal{F}), (R, \mathcal{B})$ be measure spaces, and let $X:\Omega 	o R$ be a measurable function. Suppose that $\mathcal{B}_0$ is a generating set for $\mathcal{B}$. We then have $\left\langle X^{-1}(\mathcal{B}_0) ightangle = X^{-1}\left(\langle \mathcal{B}_0 angle ight).$
\footnote{Proof of the Lemma 1:
\begin{proof}
We use the usual trick of demonstrating that the set on the left-hand side is both the subset and the superset of the set on the right-hand side.
\begin{itemize}
\item[$\subset$] From $\mathcal{B}_0\subseteq \mathcal{B}$ we immediate that $X^{-1}(\mathcal{B}_0) \subseteq X^{-1}(\mathcal{B})$. If $X^{-1}(\mathcal{B})$ would be a $\sigma$-algebra itself, we could conclude that $\langle X^{-1}(\mathcal{B}_0) angle \subseteq X^{-1}(\mathcal{B})$. Thus, we verify that $X^{-1}(\mathcal{B})$ is a $\sigma$-algebra.
\begin{enumerate}
\item[$\sigma_1$] We have $\emptyset = X^{-1}(\emptyset)$ for $\emptyset \in \mathcal{B}$.; thus, $\emptyset \in X^{-1}(\mathcal{B})$.
\item[$\sigma_2$] Let $E, F$ be a pair of sets in $\mathcal{B}$, i.e., $E = X^{-1}(A)$ and $F = X^{-1}(B)$ for some $A,B \in \mathcal{B}$. Also, $E \setminus F = X^{-1}(A) \setminus X^{-1}(B) = X^{-1}(A \setminus B)$ and $A \setminus B \in \mathcal{B}$; thus, $E \setminus F \in X^{-1}(\mathcal{B})$.
\item[$\sigma_3$] Let $E_1,E_2,\ldots$ be a sequence of sets in $X^{-1}(\mathcal{B})$, i.e., there exists a sequence $B_1,B_2,\ldots$ of sets in $\mathcal{B}$ such that $E_n = X^{-1}(A_n)$. Then $\bigcup_{n=1}^{\infty} E_n = \bigcup_{n=1}^{\infty}X^{-1}(A_n) = X^{-1}\left(\bigcup_{n=1}^{\infty}A_night)$. Since the last term is contained in $\mathcal{B}$, its inverse image under $X$ must be contained in $\mathcal{B}$.
\end{enumerate}

\item[$\supset$] We want to demonstrate that $X^{-1}(\mathcal{B}) \subseteq \langle X^{-1}(\mathcal{B}) angle$. The trick is to look at the following set
$$\mathcal{B}' := \left\{ B \in \mathcal{B} \mid X^{-1}(B) \in \langle X^{-1}(\mathcal{B}) angle ight\}.$$
If our assertion is true, this set must be exactly the whole $\mathcal{B}$. We make use of the fact that the above is a $\sigma$-algebra.
\begin{enumerate}
\item[$\sigma_1$] Empty set is contained in this set as $X^{-1}(\emptyset) = \emptyset \in \langle X^{-1}(\mathcal{B}) angle$.
\item[$\sigma_2$] Let $A,B \in \mathcal{B}'$. Then $X^{-1}(A\setminus B) = X^{-1}(A) \setminus X^{-1}(B) \in \langle X^{-1}(\mathcal{B}) angle$ and so $A\setminus B \in \mathcal{B}'$.
\item[$\sigma_3$] Let $E_1,E_2,\ldots$ be a countable sequence of sets. Then, $X^{-1}\left(\bigcap E_night) = \bigcap X^{-1}(E_n) \in \langle X^{-1}(\mathcal{B}) angle$  and so $\bigcup E_n \in \mathcal{B}'$.
\end{enumerate}
Thus $\mathcal{B}'$ is a $\sigma$-algebra. But $\mathcal{B}'$ obviously contains $\mathcal{B}_0$, and thus it must also contain the smallest $\sigma$-algebra generated by it - $\mathcal{B}$. Thus, $\mathcal{B} = \mathcal{B}'$, which is simply another way of saying that $X^{-1}(\mathcal{B}) \subseteq \langle X^{-1}(\mathcal{B}) angle$.
\end{itemize}
\end{proof}
}
\end{lemma}

Combined with the \href{./exercise_0-11}{principle of measurable induction (Exercise 11)}, it suffices to demonstrate the theorem assertion for some generating set $\mathcal{B}_0$ of $\mathcal{B}$, instead of demonstrating it for every $\mathcal{B}$-measurable set, i.e.,
\begin{equation}
\forall S_1 \in \mathcal{B}_0,\ \forall S_2 \in \mathcal{B}_0: \quad {\bf P}(X \in S_1, Y \in S_2) = {\bf P}(X \in S_1) \cdot {\bf P}(Y \in S_2).
\label{eq:stage0}
\end{equation}
But setting $\mathcal{B}_0$ to be the set $\{(-\infty,t]\mid t\in\mathbf{R}\}$ of all half-open intervals (see \href{../an_introduction_to_measure_theory/exercise_1-4-14}{Exercise 1.4.14 from Measure theory}), we automatically obtain the assertion directly from the assumption.
\begin{equation}
\forall t_1 \in \mathbf{R},\ \forall t_2 \in \mathcal{B}_0: \quad {\bf P}(X \leq t_1, Y \leq t_2) = {\bf P}(X \leq t_1) \cdot {\bf P}(Y \leq t_2).
\label{eq:stage1}
\end{equation}
Thus, it is only left to verify that independence constitutes a $\sigma$-property. We do so separately in each variable, starting with $Y$.
\begin{enumerate}
\item[$\sigma_1$] We have $0 = {\bf P}(X \leq t_1, Y \in \emptyset) = {\bf P}(X \leq t_1) \cdot {\bf P}(Y \in \emptyset) = {\bf P}(X \leq t_1) \cdot 0 = 0$.
\item[$\sigma_2$] Suppose that ${\bf P}(X \leq t_1,\, Y \in A) = {\bf P}(X \leq t_1) {\bf P}(Y \in A)$ is true for some $A$. We then have, by finite additivity of ${\bf P}$, that:
\begin{align*}
{\bf P}\left(X \leq t_1ight) \cdot {\bf P}\left(Y \in A^cight) &= {\bf P}\left(X \leq t_1ight) \cdot \left[1 - {\bf P}\left(Y \in Aight)ight]\
&= {\bf P}\left(X \leq t_1ight) - {\bf P}\left(X \leq t_1ight)\cdot {\bf P}\left(Y \in Aight)\
&= {\bf P}\left(X \leq t_1ight) - {\bf P}\left(X \leq t_1,\, Y \in Aight)\
&= {\bf P}\left(X \leq t_1,\, Y 
ot\in Aight)
\end{align*}
\item[$\sigma_3$] Let $A_1,A_2,\ldots$ be sets such that ${\bf P}(X \leq t_1,\, Y \in A_n) = {\bf P}(X \leq t_1) \cdot {\bf P}(Y \in A_n)$. Performing the usual trick of disjointisation $\bigcup_{n=1}^{\infty} A_n = \bigcup_{n=1}^{\infty} A_n \setminus \bigcup_{i=1}^{n-1} A_i$ and applying countable additivity of $P$ twice, we obtain:
\begin{align*}
{\bf P}\left(X \leq t_1,\, Y \in \bigcup_{n=1}^{\infty} A_night) &= {\bf P}\left(X \leq t_1,\, Y \in \bigcup_{n=1}^{\infty} A_n \setminus \bigcup_{i=1}^{n-1} A_i ight)\
&= \sum_{n=1}^{\infty} {\bf P}\left(X \leq t_1,\, Y \in A_n \setminus \bigcup_{i=1}^{n-1} A_i ight)\
&= \sum_{n=1}^{\infty} {\bf P}\left(X \leq t_1ight) \cdot {\bf P}\left(Y \in A_n \setminus \bigcup_{i=1}^{n-1} A_i ight)\
&= {\bf P}\left(X \leq t_1ight) \cdot \sum_{n=1}^{\infty} {\bf P}\left(Y \in A_n \setminus \bigcup_{i=1}^{n-1} A_i ight)\
&= {\bf P}\left(X \leq t_1ight) \cdot {\bf P}\left(Y \in \bigcup_{n=1}^{\infty}A_n \setminus \bigcup_{i=1}^{n-1} A_i ight)\
&= {\bf P}\left(X \leq t_1ight) \cdot {\bf P}\left(Y \in \bigcup_{n=1}^{\infty}A_night)
\end{align*}
\end{enumerate}
This closes the induction and we can safely upgrade Assertion \eqref{eq:stage1} from generating set to the whole $\sigma$-algebra:
\begin{equation}
\forall t_1 \in \mathbf{R},\ \forall S_2 \in \mathcal{B}: \quad {\bf P}(X \leq t_1,\, Y \in S_2) = {\bf P}(X \leq t_1) \cdot {\bf P}(Y \in S_2).
\label{eq:stage2}
\end{equation}
Performing the exact same operations of measurable induction on $X$, we obtain
\begin{equation}
\forall S_1 \in \mathcal{B},\ S_2 \in \mathcal{B}: \quad {\bf P}(X \in S_1,\, Y \in S_2) = {\bf P}(X \in S_1) \cdot {\bf P}(Y \in S_2).
\label{eq:stage3}
\end{equation}
By inducting on $n\in\mathbf{N}$ we can generalize this result to any number of jointly independent random variables $X_1,\ldots, X_n$.
\end{itemize}

Exercise 2.21 (Independence of real-valued random variables)

Answers

Comments

Exercise 2.21 (Independence of real-valued random variables)

Answers

Comments

Add answer