Homepage › Solution manuals › Yaser Abu-Mostafa › Learning from Data › Exercise 9.7

Learning from Data

Exercise 9.7

Answers

(a)

\begin{array}{l} z & = [\begin{matrix} x^{T} v_{1} \\ \dots \\ x^{T} v_{k} \end{matrix}] \\ = [\begin{matrix} v_{1}^{T} x \\ \dots \\ v_{k}^{T} x \end{matrix}] \\ = [\begin{matrix} v_{1}^{T} \\ \dots \\ v_{k}^{T} \end{matrix}] x \\ = V^{T} x \end{array}

The dimension of $V$ is $d \times k$ , its columns are the basis $v_{1}, \dots, v_{d}$

(b)

\begin{array}{l} Z & = [\begin{matrix} z_{1}^{T} \\ \dots \\ z_{N}^{T} \end{matrix}] \\ = [\begin{matrix} {(V^{T} x_{1})}^{T} \\ \dots \\ {(V^{T} x_{N})}^{T} \end{matrix}] \\ = [\begin{matrix} x_{1}^{T} V \\ \dots \\ x_{N}^{T} V \end{matrix}] \\ = [\begin{matrix} x_{1}^{T} \\ \dots \\ x_{N}^{T} \end{matrix}] V \\ = XV \end{array}

(c)

We consider the

z = V^{T} x

where

z

has

d

components, so

V

is

d \times d

.

V = [\begin{matrix} v_{1}^{T} \\ \dots \\ v_{d}^{T} \end{matrix}]

, since

v

s are othrogonal, we have

V^{T} V = I

, so

V^{T} = V^{- 1}

\begin{array}{l} \sum_{i = 1}^{d} z_{i}^{2} & = ∥ z ∥^{2} \\ = z^{T} z \\ = {(V^{T} x)}^{T} (V^{T} x) \\ = x^{T} V V^{T} x \\ = x^{T} V V^{- 1} x \\ = x^{T} x \\ = \sum_{i = 1}^{d} x_{i}^{2} \end{array}

If we choose $k \leq d$ for $z$ , we thus have $∥ z ∥ \leq ∥ x ∥$ .

niuers

2021-12-08 10:24

Comments