Exercise 2.20 (Independence of countable-valued random variables)

Question

Let $X_1,\dots,X_n$ be discrete random variables (i.e. they take values in at most countable spaces $R_1,\dots,R_n$ equipped with the discrete sigma-algebra). Show that $X_1,\dots,X_n$ are jointly independent if and only if one has

$$\displaystyle {\bf P}\left( \bigwedge_{i=1}^n (X_i=x_i) \right) = \prod_{i=1}^n {\bf P}(X_i = x_i)$$

for all $x_1 \in R_1,\dots, x_n \in R_n$.

Elu Thingol · Accepted Answer

To demonstrate the equivalence of both conditions, we show that the implications hold in both directions.

\begin{itemize}
\item[$\implies$] The 	extit{only if} part of the proof is obvious: Assume that the random variables $X_1,\ldots,X_n$ are jointly independent; as per Definition 16 (Independence), set $S_i := \{x_i\}$. We then trivially have
\begin{align*}
{\bf P}\left( \bigwedge_{i=1}^n (X_i=x_i) ight) &= {\bf P}\left( \bigwedge_{i=1}^n (X_i \in S_i) ight)\
&= \prod_{i=1}^{n} {\bf P}\left( X_i \in S_i ight)\
&= \prod_{i=1}^{n} {\bf P}\left( X_i = x_i ight).
\end{align*}

\item[$\impliedby$] Now assume that the converse holds. For the product variable $X=(X_1,\ldots,X_n)$ and a random $S_1	imes \ldots 	imes S_n$ for $S_i\in\mathcal{B}_i$, we have, by additivity of the probability measure, that
\begin{align*}
P(X \in S)
&= \sum_{x \in S} P(X = x)\
&= \sum_{x_1 \in S_1, \ldots, x_n \in S_n} P\left(X_1 = x_1, \ldots, X_n = x_night)
\intertext{We apply the theorem assumption, tranforming the probability law of the product into the product of probability laws:}
&= \sum_{x_1 \in S_1, \ldots, x_n \in S_n} \prod_{i=1}^{n} P\left(X_i = x_iight)
\intertext{It is easy to verify that we can interchange sums and products in this case\footnote{We induct on $n$. \begin{itemize}
\item For the induction base, we have $$\sum_{a \in A} \sum_{b \in B} f(a)\cdot f(b) = \sum_{a \in A} \left(f(a) \cdot \sum_{b \in B} f(b)ight) = \left(\sum_{b \in B} f(b)ight) \cdot \left(\sum_{a \in A} f(a) ight)$$.
\item The induction step follows similarly.
\begin{align*}
\sum_{x_1 \in S_1} \ldots \sum_{x_n \in S_n} \sum_{x_{n+1} \in S_{n+1}} f(x_1) \cdot \ldots \cdot f(x_n) \cdot f(x_{n+1})
&= \sum_{x_1 \in S_1} \cdot \ldots \cdot \sum_{x_n \in S_n} f(x_1) \cdots f(x_n) \left[\sum_{x_{n+1} \in S_{n+1}} f(x_{n+1})ight]\
&= \left[\prod_{i=1}^{n} \sum_{x_i \in S_i} f(x_i) ight]\cdot \left[\sum_{x_{n+1} \in S_{n+1}} f(x_{n+1})ight]\
&= \prod_{i=1}^{n+1} \sum_{x_i \in S_i} f(x_i).
\end{align*}
This closes the induction.
\end{itemize}
}}
&= \prod_{i=1}^{n} \sum_{x_1 \in S_1, \ldots, x_n \in S_n} P\left(X_i = x_iight)
\intertext{By countable additivity, we perform the reverse step and obtain the desired result}
&= \prod_{i=1}^{n} P\left(X_i \in S_iight).
\end{align*}

\end{itemize}

Exercise 2.20 (Independence of countable-valued random variables)

Answers

Comments

Exercise 2.20 (Independence of countable-valued random variables)

Answers

Comments

Add answer