Exercise 6.12

Answers

(a)

For nonparametric RBF,

g (x)

is always a weighted sum of

y_{n}

, as

| x | \to \infty

, all

| x - x_{n} |

are equal to each other, so the weights are uniform, in such case,

g (x) = \frac{1}{N} \sum y_{n}

, the average of

y_{n}

For the parametric RBF, each $Φ_{n} (x) \to 0$ when $| x | \to \infty$ , and the weights $w_{n}$ are constants, so the RBF goes to 0 as well.

(b)

Note

Z_{nj} = Φ_{j} (x_{n})

Z

is a

N \times N

matrix, where its

n

th row is

Φ^{T} (x_{n}) = [\begin{matrix} Φ_{1} (x_{n}) & Φ_{2} (x_{n}) & \dots & Φ_{n} (x_{n}) & \dots & Φ_{N} (x_{n}) \end{matrix}] .

So its $n$ th column is just $Φ (x_{n})$ . So we have

Z^{T} = [\begin{matrix} Φ (x_{1}) & Φ (x_{2}) & \dots & Φ (x_{n}) & \dots & Φ (x_{N}) \end{matrix}] .

If $Z$ is invertible, we have ${(Z^{T})}^{- 1} Z^{T} = I$ , and

{(Z^{T})}^{- 1} Φ (x_{n}) = [\begin{matrix} 0 \\ \dots \\ 0 \\ 1 \\ \dots \\ 0 \end{matrix}],

where the only 1 is at the $n$ th position.

So we obtain

g (x_{n}) = w^{T} Φ (x_{n}) = {(Z^{- 1} y)}^{T} Φ (x_{n}) = y^{T} Z^{- T} Φ (x_{n}) = y^{T} [\begin{matrix} 0 \\ \dots \\ 0 \\ 1 \\ \dots \\ 0 \end{matrix}] = y_{n} .

So $g (x)$ exactly interpolates the data points.

(c)

The nonparametric RBF does NOT always have

E_{in} = 0

g (x)

is always a weighted sum of

y_{n}

, at

x = x_{k}

g (x_{k})

is still a weighted sum of

y_{n}

s, so we don’t have

g (x_{k}) = y_{k}

here.

niuers

2021-12-08 09:48