Exercise 4.11 - Derivation of the NIW posterior

We begin with the likelihood for a MVN:

By (4.195), which can be proven by:

where we have used the fact that $\text{tr}({\mathbf{\text{𝐘}}}^{\text{T}}\mathbf{\text{𝐙}})=\text{tr}(\mathbf{\text{𝐙}}{\mathbf{\text{𝐘}}}^{\text{T}})$, with $\mathbf{\text{𝐘}}$ the shifted design matrix and $\mathbf{\text{𝐙}}={\mathrm{\Sigma }}^{-1}\mathbf{\text{𝐘}}$.

The conjugate prior for MVN’s parameters $(\mu ,\mathrm{\Sigma })$ is Normal-inverse-Wishart(NIW) distribution defined by:

Hence the posterior reads (where we have omitted the condition on hyperparameters):

where $v_{\mathbf{\text{𝐗}}}$, $k_{\mathbf{\text{𝐗}}}$, ${\mathbf{\text{𝐦}}}_{\mathbf{\text{𝐗}}}$ and ${\mathbf{\text{𝐒}}}_{\mathbf{\text{𝐗}}}$ are variables whose values are to be decided. Only terms that dependent on $\mu$ and $\mathrm{\Sigma }$ can explicitly enter the terms on the r.h.s.

Firstly, by comparing the exponential for $|\mathrm{\Sigma }|$, we have:

Secondly, compare the coefficient for the term ${\mu }^{\text{T}}{\mathrm{\Sigma }}^{-1}\mu$ inside the exponential and we have:

Thirdly, check the coefficient for ${\mu }^{\text{T}}$ so we have:

therefore:

Finally, recall that for an arbitrary column vector $A$:

The terms that solely dependent on ${\mathrm{\Sigma }}^{-1}$ should equal to each other, so:

Having arrived in:

we obtain:

Recall the definition for mean we ends in (4.214) since:

This finishes proving that the posterior distribution for MVN takes the form: $\text{NIW}({\mathbf{\text{𝐦}}}_{\mathbf{\text{𝐗}}},k_{\mathbf{\text{𝐗}}},v_{\mathbf{\text{𝐗}}},{\mathbf{\text{𝐒}}}_{\mathbf{\text{𝐗}}}).$