Exercise 4.4

Let’s compute $E_{\mathit{in}}(w)$:

On the other hand, the equation (4.3) is

This agrees with the result derived above. So we proved equation (4.3).

(a)

The value of $w$ that minimizes $E_{\mathit{in}}$ is $w_{\mathit{lin}}$, since it makes the first term $0$, which is a quadratic term on $w$ and is greater or equal to $0$ all the time. The second term doesn’t depend on $w$.

(b)

The minimum in-sample error when $w=w_{\mathit{lin}}$ is simply the second term, i.e. $\min <!--\; FUNCTION\; APPLICATION\; -->E_{\mathit{in}}(w)=\frac{y^T(I-H)y}{N}$.