Exercise 8.17

With classification 0/1 error, $E_{\mathit{in}}=\frac{1}{N}{\sum <!--\; FUNCTION\; APPLICATION\; -->}_{n=1}^N[\mathit{sign}(w^T{x}_n+b)\ne y_n]$ and $E_{\mathit{SV}M}(b,w)=\frac{1}{N}{\sum <!--\; FUNCTION\; APPLICATION\; -->}_{n=1}^N\mathit{max}(1-y_n(w^T{x}_n+b),0)$.

For any $n$,

• If the point is correctly classified with no violation, then $y_n(w^T{x}_n+b)\ge 1$ and

  $$\max <!--\; FUNCTION\; APPLICATION\; -->(1-y_n(w^T{x}_n+b),0)=0,$$

  so $e_{\mathit{in}}=e_{\mathit{SV}M}$

• If the point is correctly classified with violation, then $1>y_n(w^T{x}_n+b)\ge 0$, so

  $$\mathit{max}(1-y_n(w^T{x}_n+b),0)=1-y_n(w^T{x}_n+b),$$

  so we have $0=e_{\mathit{in}}<e_{\mathit{SV}M}$

• If the point is wrongly classified, then we have $y_n(w^T{x}_n+b)<0$ and

  $$\mathit{max}(1-y_n(w^T{x}_n+b),0)=1-y_n(w^T{x}_n+b)>1,$$

  so we have $1=e_{\mathit{in}}<e_{\mathit{SV}M}$

We conclude that $E_{\mathit{in}}\le E_{\mathit{SV}M}$