Exercise 4.9.1

We form the square matrix $Y^{T}X=X=\left[\begin{array}{cc}\hfill 1\hfill & \hfill 2\hfill \\ \hfill 2\hfill & \hfill 1\hfill \end{array}\right]$, we need find the SVD for this matrix.

let $K=Y^{T}X$, then we have $K^{T}K=\left[\begin{array}{cc}\hfill 5\hfill & \hfill 4\hfill \\ \hfill 4\hfill & \hfill 5\hfill \end{array}\right]$, the eigenvalues are ${\lambda }_1=9,{\lambda }_2=1$, with eigenvectors: $v_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \end{array}\right]$, and $v_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill -1\hfill \end{array}\right]$, these are the right singular vectors as well, The corresponding left singular vectors are $u_1=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill 1\hfill \\ \hfill 1\hfill \end{array}\right]$ and $u_2=\frac{1}{\sqrt{2}}\left[\begin{array}{c}\hfill -1\hfill \\ \hfill 1\hfill \end{array}\right]$, in the end we have

$K=U\mathrm{\Sigma }{V}^T=\frac{1}{\sqrt{2}}\left[\begin{array}{cc}\hfill 1\hfill & \hfill -1\hfill \\ \hfill 1\hfill & \hfill 1\hfill \end{array}\right]\left[\begin{array}{cc}\hfill 3\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right]\frac{1}{\sqrt{2}}\left[\begin{array}{cc}\hfill 1\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill -1\hfill \end{array}\right]$.

The orthogonal matrix $Q$ that minimizes the norm is $Q=U{V}^T=\left[\begin{array}{cc}\hfill 0\hfill & \hfill 1\hfill \\ \hfill 1\hfill & \hfill 0\hfill \end{array}\right]$, the minimal nor is: 4.

If we let $Q=\left[\begin{array}{cc}\hfill \cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill -\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \\ \hfill \sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill \cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \end{array}\right]$, then we have

$$\begin{array}{llll}\hfill \parallel X-YQ{\text{∥}}_F^2& =\parallel X-Q{\text{∥}}_F^2& \hfill & \\ \hfill & =\parallel \left[\begin{array}{cc}\hfill 1\hfill & \hfill 2\hfill \\ \hfill 2\hfill & \hfill 1\hfill \end{array}\right]-\left[\begin{array}{cc}\hfill \cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill -\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \\ \hfill \sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill \cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \end{array}\right]{\text{∥}}_F^2& \hfill & \\ \hfill & =\parallel \left[\begin{array}{cc}\hfill 1-\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill 2+\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \\ \hfill 2-\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill & \hfill 1-\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃\hfill \end{array}\right]{\text{∥}}_F^2& \hfill & \\ \hfill & ={(1-\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃)}^2+{(2+\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃)}^2+{(2-\sin <!--\; FUNCTION\; APPLICATION\; -->𝜃)}^2+{(1-\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃)}^2& \hfill & \\ \hfill & =2{(1-\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃)}^2+2(4+{\sin <!--\; FUNCTION\; APPLICATION\; -->}^2𝜃)& \hfill & \\ \hfill & =2(1-2\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃+{\cos <!--\; FUNCTION\; APPLICATION\; -->}^2𝜃+4+{\sin <!--\; FUNCTION\; APPLICATION\; -->}^2𝜃)& \hfill & \\ \hfill & =2(6-2\cos <!--\; FUNCTION\; APPLICATION\; -->𝜃)& \hfill & \\ \hfill & & \hfill \end{array}$$

This is minimized when $𝜃=2\mathit{\pi k}$, where $k=0,1,\dots$

So $Q=\left[\begin{array}{cc}\hfill 1\hfill & \hfill 0\hfill \\ \hfill 0\hfill & \hfill 1\hfill \end{array}\right]$, the minimal norm is 8. It misses the optimal $Q$ computed above, which is caused by the assumption that the off diagonal entries are opposite of each other.

import numpy  as np 
u = np.array([[1, -1], [1, 1]]) 
v = np.array([[1, 1], [1, -1]]) 
m = np.array([[3, 0], [0, 1]]) 
np.matmul(np.matmul(u, m), v.transpose()) 
np.matmul(v.transpose(), u)

array([[ 2,  0],
       [ 0, -2]])