Exercise 4.14 - MAP estimation for 1d Gaussians

Assume that the variance for this distribution ${\sigma }^2$ is known, and the mean $\mu$ is subject to a normal distribution with mean $m$ and variance $s^2$. Similiar to the question before, the posterior takes the form:

So the posterior is another normal distribution, by comparing the coefficient for ${\mu }^2$ in the exponential:

and that for $\mu$:

we have the posterior mean and variance by completing the square:

This finishes question (a).

For question (b), we already knew that the MLE is:

As $N$ increases, we have:

For question (c), when $s^2\to \infty$, ${\mu }_{\text{post}}$ also converges to ${\mu }_{\text{MLE}}$ since $\frac{{\sigma }^2}{s^2}\to 0$.

For question (d), when $s^2\to 0$, then $\frac{{\sigma }^2}{s^2}\to \infty$ and ${\mu }_{\text{post}}$ converges to $m$.

Both (c) and (d) are very intuitive. $s^2\to \infty$ means a non-informative prior has been introduced so MAP is the same as MLE. $s^2\to 0$ means that the knowledge that $\mu$ is close to $m$ is very strong so that finite observations cannot modify this belief.