Homepage › Solution manuals › Yaser Abu-Mostafa › Learning from Data › Exercise 3.3

Learning from Data

Exercise 3.3

Answers

(a)

H

is symmetric.

\begin{array}{l} H^{T} & = {(X {(X^{T} X)}^{- 1} X^{T})}^{T} \\ = X {(X^{T} X)}^{- T} X^{T} \\ = X {(X^{T} X)}^{- 1} X^{T} \\ = H \end{array}

(b)

We show that

H^{2} = H

first.

\begin{array}{l} H^{2} & = (X {(X^{T} X)}^{- 1} X^{T}) (X {(X^{T} X)}^{- 1} X^{T}) \\ = X {(X^{T} X)}^{- 1} (X^{T} X) {(X^{T} X)}^{- 1} X^{T} \\ = X {(X^{T} X)}^{- 1} X^{T} \\ = H \end{array}

Apply the above relationship repeatedly for $H^{K}$ we see that $H^{K} = H$ .

(c)

First show

{(I - H)}^{2} = I - H

.

\begin{array}{l} {(I - H)}^{2} & = (I - H) (I - H) \\ = II - IH - HI + H^{2} \\ = I - 2 H + H^{2} \\ = I - 2 H + H \\ = I - H \end{array}

Apply the above relationship repeatedly for ${(I - H)}^{K}$ , we have ${(I - H)}^{K} = I - H$ .

(d)

Let

A = X {(X^{T} X)}^{- 1}

,

B = X^{T}

, apply the property

trace (AB) = trace (BA)

, we have

\begin{array}{l} trace (H) & = trace (X {(X^{T} X)}^{- 1} X^{T}) \\ = trace (AB) \\ = trace (BA) \\ = trace (X^{T} X {(X^{T} X)}^{- 1}) \\ = trace (I) \\ = d + 1 \end{array}

where the identity matrix

I

is of size

d + 1

.

niuers

2021-12-07 22:12

Comments