Homepage › Solution manuals › Yaser Abu-Mostafa › Learning from Data › Exercise 7.7

Exercise 7.7

Answers

$\tanh (x) = \frac{e^{x} - e^{- x}}{e^{x} + e^{- x}}$ , we have $\frac{d \tanh (x)}{dx} = \frac{4}{{(e^{x} + e^{- x})}^{2}} = 1 - \tanh^{2} (x)$ , so we have

\begin{array}{l} \nabla E_{in} (w) & = \nabla \frac{1}{N} \sum_{n = 1}^{N} {(\tanh (w^{T} x_{n}) - y_{n})}^{2} \\ = \frac{1}{N} \sum_{n = 1}^{N} \nabla {(\tanh (w^{T} x_{n}) - y_{n})}^{2} \\ = \frac{1}{N} \sum_{n = 1}^{N} 2 (\tanh (w^{T} x_{n}) - y_{n}) \frac{d \tanh (w^{T} x_{n})}{dx} \\ = \frac{1}{N} \sum_{n = 1}^{N} 2 (\tanh (w^{T} x_{n}) - y_{n}) (1 - \tanh^{2} (w^{T} x_{n})) x_{n} \\ = \frac{2}{N} \sum_{n = 1}^{N} (\tanh (w^{T} x_{n}) - y_{n}) (1 - \tanh^{2} (w^{T} x_{n})) x_{n} \end{array}

If $w \to \infty$ , $\tanh (w^{T} x_{n}) \to 1$ , and $\nabla E_{in} (w) \to 0$ , so the gradient won’t change when $w \to \infty$ , which is hard to improvoe the perceptron solution.

niuers

2021-12-08 09:53

Exercise 7.7

Answers

Comments

Add answer