Exercise 6.10

Answers

(a): When $k = 1$ , we have $g (x) = y_{[1]} (x)$ , at each data point $x = x_{i}$ , we have $g (x_{i}) = y_{[1]} (x_{i}) = y (x_{i})$ , so $E_{in} = \sum | g (x_{i}) - y (x_{i}) |^{2} = 0$
(b): The final hypothesis is not smooth, because as we increase $x$ , their nearest neighborhoods is changing, within an interval, for all $x$ in that interval, their $k$ nearest neighbors are fixed, so we have constant average from these $k$ nearest neighbors for $g (x)$ within the interval. But once we move out of the interval, new neighbor joined and one old neighbor dropped, the average of the neighborhood immediately changed, so we see the step-wise pattern in the final hypothesis.
(c): If $k$ is very small, say, 1, 2, the final hypothesis $g$ is very unsmooth, it can match the data points closely (or exactly if $k = 1$ ), but the approximation to $f$ can be very bad because it tries too hard to fit the noises.

If $k$ is very large, the final hypothesis $g$ approaches the average of the whole data set, which is a constant, this is too simplistic to approximate the target function $f$ .
(d): If $x \to \infty$ , or $x \to - \infty$ , the nearest neighbors for $x$ will be a fixed set of points, i.e. the data points at two ends. So the hypothesis will generate constant values for $x$ going large at both ends.

niuers

2021-12-08 09:46