Exercise 6.2

Given a test point $(x,y)$ (Generated from $P(x)$ and $\pi (x)=P[y=1|x]$) and the target function $f(x)$, The probability of error on a test point $x$ is:

For any other hypothesis $h$ (deterministic or not), we apply the same calculation, and have

Where $P_1=P(h(x)=1)$, If $\pi (x)\ge \frac{1}{2}$, then $1-2\pi (x)\le 0$, and $\pi (x)+(1-2\pi (x))P_1\ge \pi (x)+(1-2\pi (x))=1-\pi (x)$, since $P_1\le 1$.

If $\pi (x)<\frac{1}{2}$, then $1-2\pi (x)>0$, and $\pi (x)+(1-2\pi (x))P_1\ge \pi (x)$, since $P_1\ge 0$.

So we conclude that: $e(h(x))\ge e(f(x))$.