Exercise 7.16

In the picture, we have 2-Dimensional components in the gradient of $E_{\mathit{in}}$. At $w=w(t+1)$, it achieves minimum along the direction $v(t)$, so one component of the gradient of $E_{\mathit{in}}$ is zero. Since $v(t+1)$ is the conjugate direction of $v(t)$, when we move along $v(t+1)$, the gradient along this direction will remain perpendicular to the previous search direction $v(t)$, so the point that makes the second component zero is on this line. This means there’s one point that have both gradients equal to zero. That point is the optimal weights.