Exercise 3.6

(a) The probability to get $y_n$ is $P(y_n|x_n)$, by maximum likelihood method, we need maximize the likelihood, ${\prod <!--\; FUNCTION\; APPLICATION\; -->}_{n=1}^{N}P(y_n|x_n)$, this is equivalent to maximize the logrithm of it: ${\sum <!--\; FUNCTION\; APPLICATION\; -->}_{n=1}^N\ln <!--\; FUNCTION\; APPLICATION\; -->(P(y_n|x_n))$, or minimize the negative of it: $-{\sum <!--\; FUNCTION\; APPLICATION\; -->}_{n=1}^N\ln <!--\; FUNCTION\; APPLICATION\; -->(P(y_n|x_n))$

When $y_n=+1$, $P(y_n|x_n)=h(x_n)$, and when $y_n=-1$, $P(y_n|x_n)=1-h(x_n)$, separate the cases for $y_n=1$ and $y_n=-1$, we have:

(b) For $h(x)=𝜃(w^{T}x)=\frac{e^{w^{T}x}}{1+e^{w^{T}x}}$, we have $\ln <!--\; FUNCTION\; APPLICATION\; -->\frac{1}{h(x_n)}=\ln <!--\; FUNCTION\; APPLICATION\; -->(1+e^{-w^T{x}_n})$ and $\ln <!--\; FUNCTION\; APPLICATION\; -->\frac{1}{(1-h(x_n))}=\ln <!--\; FUNCTION\; APPLICATION\; -->(1+e^{w^T{x}_n})$. Combine them together we have

Which is equivalent to minimizing the one in equation (3.9).