Exercise 11.5 - Gradient descent for fitting GMM

From the given (11.118) and (11.119):

While $r_{\mathit{𝑛𝑘}}=p(z_{\mathit{𝑛𝑘}}=1|{\mathbf{\text{𝐱}}}_n,𝜃)$ is defined by (11.120).

For question (a), recall that:

We are now ready to taking partial gradient of $l$ w.r.t. ${\mu }_k$, which yields:

using (4.10) for the last step. Now we have arrived in (11.121).

For question (b):

For question (c), with:

we have:

where no constant factor is missed.

For question (d), we have:

during which process we need to use (4.10) and the fact:

for a symmetric matrix $\mathbf{\text{𝐀}}$. Thus the optimal ${\mathrm{\Sigma }}_k$ takes the same form as what has been derived in exercise 11.2.

For question (e), this process is redundant since the MLE for $\mathrm{\Sigma }$ is already a positive definite matrix.