Exercise 3.2.7

• (1)

  The derivative of a singular value is $\frac{\mathit{d\sigma }}{\mathit{dt}}=u^T(t)\frac{\mathit{dA}}{\mathit{dt}}v(t)$, so we have

$\frac{\mathit{dA}(0)}{\mathit{dt}}=A$, At $t=0$, we have the matrix $D+\mathit{tA}=D$, so we here look for the singular values and vectors of $D$.

Since $D=\mathit{IDI}$, its singular values are: $1,2,\dots ,n$, the left and right singular vectors are $x_i=(0,0,\dots ,1,0,\dots ,0)$ where the 1 is at the $i$th position.

So at a singular value ${\sigma }_i=i$, we have $\frac{\mathit{d\sigma }}{\mathit{dt}}=\left[\begin{array}{cccccc}\hfill 0\hfill & \hfill \dots \hfill & \hfill 1\hfill & \hfill 0\hfill & \hfill \dots \hfill & \hfill 0\hfill \end{array}\right]A\left[\begin{array}{c}\hfill 0\hfill \\ \hfill ⋮\hfill \\ \hfill 1\hfill \\ \hfill 0\hfill \\ \hfill ⋮\hfill \\ \hfill 0\hfill \end{array}\right]=A_{\mathit{ii}}$.

The derivative at $t=0$ of the singular values $\sigma (t)$ is the diagonal values of $A$.

• (1)

  From Weyl’s inequalities we have ${\sigma }_{\mathit{max}}(D+\mathit{tA})={\sigma }_1(D+\mathit{tA})\le {\sigma }_1(D)+{\sigma }_1(\mathit{tA})=n+t{\sigma }_1(A)$

and ${\sigma }_{\mathit{min}}(D+\mathit{tA})={\sigma }_n(D+\mathit{tA})\le {\sigma }_i(D)+{\sigma }_j(\mathit{tA})=(n+1-i)+t{\sigma }_j(A)$. Also we need $i+j=n+1$, so ${\sigma }_{\mathit{min}}(D+\mathit{tA})\le (n+1-i)+t{\sigma }_j(A)=j+t{\sigma }_j(A)$

#### Problem III.2.8

• 1.

  Because subspace $V$ has a dimension of $i$, and subpsace $Z$ has a dimension of $n-i+1$, they must have an intersection, otherwise let $p={\sum <!--\; FUNCTION\; APPLICATION\; -->}_{k=1}^i{a}_k{v}_k$ a vector in $V$ and $q={\sum <!--\; FUNCTION\; APPLICATION\; -->}_{k=i}^n{b}_k{q}_k$ a vector in $Z$, if $V$ and $Z$ doesn’t intersect, $v_k$ and $q_k$ are all independent of each other, so we’ll have $p+q={\sum <!--\; FUNCTION\; APPLICATION\; -->}_{k=1}^{n+1}{c}_k{x}_k$, This contradicts with the fact that there are only $n$ independent basises in total.

Since $V$ intersect with $Z$, it means that there’s a nonzero vector $z$ both in $V$ and $Z$, such that $z$ can be expressed as a combination of $q_i,q_{i+1},\dots ,q_n$.

• 1.

  If ${\lambda }_i$ is the eigenvalue corresponds to the $i$th large eigenvalue of $S$, since $z$ is also in the subspace of $Z$, where the maximum eigenvalue is ${\lambda }_i$, so we must have $\frac{z^T\mathit{Sz}}{z^{T}z}\le {\lambda }_i$.

Then $\underset{z\mathit{in}V}{\min <!--\; FUNCTION\; APPLICATION\; -->}=\frac{z^T\mathit{Sz}}{z^{T}z}$ indicates that for any $i$-dimensional subspace $V$, the mimimum value can range from ${\lambda }_n,{\lambda }_{n-1},\dots ,{\lambda }_i$ because you can choose any $i$ eigenvectors to compose such space $V$.

If we then take the maximum among the $i$-dimensional subspace $V$, we see that the maximum value we can achieve is ${\lambda }_i$.