Exercise 5.7 - Bayes model averaging helps predictive accuracy

Suppose the variable $\mathrm{\Delta }$ is generated from a mixture of models, so:

The Bayes model averaging (BMA) result is just:

While that from an individual model $m\in M$ is:

The expected loss of the BMA result is:

while that of model $m$ is:

Now it is easy to see that:

since the distribution on which the expectation is computed is just $p^{\text{BMA}}(\mathrm{\Delta })$. Therefore the non-negativity of the KL divergence yields (5.127).

The conclusion from this exercise is of hardly any practical significance. Since the underlying distribution is usually intractable, even with the mixture Bayes model. Once the form of the latent distribution is revealed, it is obvious that other distributions result in higher loss.