Exercise 21.4 - Variational lower bound for VB for GMMs

For variational GMM, the lower bound is given by:

Which involved Bayes rules and the decomposition form of the posterior and the variational distribution $q$. We now proceed to prove (21.209) through (21.215).

For (21.209):

Plugging (21.131) and (21.132) into this form and we ends up with (21.209).

For (21.210):

where we used the fact the the expectation of the product of independent random variables is the product of their expectations. The notations follows (21.129).

For (21.211):

plugging (21.216) finishes the proof.

For (21.212):

Where we have used (21.131) to expand the expected value of the quadratic form and used the fact that the mean of a Wi distribution is $v_k{L}_k$.

For (21.213):

We only have to recall (21.124).

For (21.214):

Finally, for (21.215):

Using (21.132) to expand the quadratic gives $𝔼[{({\mu }_k-m_k)}^{\text{T}}({\beta }_k{\mathrm{\Lambda }}_k)({\mu }_k-m_k)]=D$.