Exercise 5.1 - Proof that a mixture of conjugate priors is indeed conjugate

Answers

The mixed conjugate prior takes the form (5.68):

p (𝜃) = \sum_{k} p (z = k) \cdot p (𝜃 | z = k),

where $k$ is the index for mixed components. Each $p (𝜃 | z = k)$ is a conjugate prior for the model, i.e., $p (𝜃 | z = k)$ and $p (𝜃 | z = k, 𝒟)$ takes the same form.

For the posterior in this case:

\begin{aligned} p (𝜃 | 𝒟) & = \sum_{k} p (𝜃, z = k | 𝒟) \\ = \sum_{k} p (z = k | 𝒟) \cdot p (𝜃 | z = k, 𝒟) . \end{aligned}

So the posterior is still a mixture of conjugate priors. This is exactly (5.69), finally:

\begin{aligned} p (z = k | 𝒟) & = \frac{p (z = k, 𝒟)}{p (𝒟)} \\ = \frac{p (z = k) \cdot p (𝒟 | p (z = k))}{\sum_{k^{'}} p (z = k^{'}) \cdot p (𝒟 | p (z = k^{'}))}, \end{aligned}

from Bayes rules. The computation bottleneck in computing $p (z = k | 𝒟)$ is:

p (𝒟 | z = k) = \int p (𝒟 | 𝜃) \cdot p (𝜃 | z = k) d 𝜃,

which is not a easy task as well.

solour_lfq

2021-03-24 13:42

Exercise 5.1 - Proof that a mixture of conjugate priors is indeed conjugate

Answers

Comments

Add answer