Exercise 12.5 - Deriving the residual error for PCA

Answers

For question (a), we have:

| | 𝐱 n − ∑ j = 1 K z 𝑛𝑗 𝐯 j | | 2 = ( 𝐱 n − ∑ j = 1 K z 𝑛𝑗 𝐯 j ) T ( 𝐱 n − ∑ j = 1 K z 𝑛𝑗 𝐯 j ) = 𝐱 n T 𝐱 n − 2 𝐱 n T ∑ j = 1 K z 𝑛𝑗 𝐯 j + ∑ j = 1 K ∑ j ′ = 1 K z 𝑛𝑗 z n j ′ 𝐯 j T 𝐯 j ′ = 𝐱 n T 𝐱 n − ∑ j = 1 K z 𝑛𝑗 2 ,

which is tantamount to (12.127).

For question (b), we have:

J K = 1 N ∑ n − 1 N ( 𝐱 n T 𝐱 n − ∑ j = 1 K z 𝑛𝑗 2 ) = 1 N ∑ n = 1 T 𝐱 n T 𝐱 n − 1 N ∑ n = 1 N ∑ j = 1 K 𝐯 j T 𝐱 n 𝐱 n T 𝐯 v = 1 N ∑ n = 1 T 𝐱 n T 𝐱 n − ∑ j = 1 K 𝐯 j T ( 1 N ∑ n = 1 N 𝐱 n 𝐱 n T ) 𝐯 j = 1 N ∑ n = 1 T 𝐱 n T 𝐱 n − ∑ j = 1 K 𝐯 j T 𝐂 𝐯 j = 1 N ∑ n = 1 T 𝐱 n T 𝐱 n − ∑ j = 1 K λ j .

For question (c), we have:

J K = 1 N ∑ n = 1 N 𝐱 n T 𝐱 n − ∑ j = 1 K λ j = 1 N ∑ n = 1 N 𝐱 n T 𝐱 n − ( ∑ j = 1 d λ j − ∑ j = K + 1 d λ j ) = 1 N ∑ n = 1 N 𝐱 n T 𝐱 n − ∑ j = 1 d λ j + ∑ j = K + 1 d λ j = J d + ∑ j = K + 1 d λ j = ∑ j = K + 1 d λ j .

User profile picture
2021-03-24 13:42
Comments