Homepage › Solution manuals › Yaser Abu-Mostafa › Learning from Data › Exercise 3.7

Exercise 3.7

Answers

Take derivative of $E_{in} (w) = \frac{1}{N} \sum_{n = 1}^{N} \ln (1 + e^{- y_{n} w^{T} x_{n}})$ with respect to $w$ , we have:

\begin{array}{l} \nabla E_{in} (w) & = - \frac{1}{N} \sum_{n = 1}^{N} \frac{y_{n} x_{n} e^{- y_{n} w^{T} x_{n}}}{1 + e^{- y_{n} w^{T} x_{n}}} \\ = - \frac{1}{N} \sum_{n = 1}^{N} \frac{y_{n} x_{n}}{1 + e^{y_{n} w^{T} x_{n}}} \\ = \frac{1}{N} \sum_{n = 1}^{N} - y_{n} x_{n} 𝜃 (- y_{n} w^{T} x_{n}) \end{array}

When a sample is misclassified, $y_{n} w^{T} x_{n} 0$ , so $𝜃 (- y_{n} w^{T} x_{n}) 0.5$ , while when a sample is correctly classified, $𝜃 (- y_{n} w^{T} x_{n}) 0.5$ , so the contribution of ’misclassified’ example is more to the gradient than a correctly classified one.

niuers

2021-12-07 22:14

Exercise 3.7

Answers

Comments

Add answer