Exercise 6.15

If we initialize ${\gamma }_{\mathit{nj}}=\frac{1}{k}$, then we have $N_j=\frac{N}{k}$, and $w_1=\cdots =w_k=\frac{1}{k}$, All the ${\mu }_j,{\mathrm{\Sigma }}_j$ are the mean and variance of the full data set, i.e. ${\mu }_j=\frac{1}{N}\sum <!--\; FUNCTION\; APPLICATION\; -->x_n$ for $j=1,\dots ,N$. So ${\mu }_1=\cdots ={\mu }_N=\mu$, and ${\mathrm{\Sigma }}_1=\cdots ={\mathrm{\Sigma }}_N=\mathrm{\Sigma }$.

The probabilities of $x_n$ given a cluster membership $j$, are all the same for different clusters, i.e. $P(x_n|S_1)=\cdots =P(x_n|S_N)=N(x_n;\mu ,\mathrm{\Sigma })$.

So the in the next iteration ${\gamma }_{\mathit{nj}}(t+1)=\frac{1}{k}={\gamma }_{\mathit{nj}}(t)$.

The $\gamma$ will stay constant in all iterations.

So in the end, all $k$ clusters, will have the same center and shape. The final probability density estimate is just a Gaussian model with data mean and variance, not a mixed of multiple Gaussian models.