Exercise 2.14 - A measure of correlation

Answers

For question (a), we only have to borrow the conclusion from Exercise 2.12.:

r = 1 H ( Y | X ) H ( X ) = H ( X ) H ( Y | X ) H ( X ) = H ( Y ) H ( Y | X ) H ( X ) = I ( X ; Y ) H ( X ) .

For question (b), we have 0 r 1 in question b for I ( X ; Y ) 0 and H ( X | Y ) 0 .

For question (c), r = 0 iff X and Y are independent so the distance between p ( x , y ) and p ( x ) p ( y ) is zero regarding KL-divergence.

For question (d), r = 1 iff X is determined by, but not necessarily equal to, Y .

User profile picture
2021-03-24 13:42
Comments