Exercise 21.3 - Variational lower bound for VB for univariate Gaussian

Recall that the variational lower bound for VB is defined by:

where $\tilde{p}$ is the (unnormalized) posterior and $q$ is the variational distribution. For the univariate Gaussian case, we already arrive in (21.84)-(21.87) and (21.92)-(21.97), thus we only have to fill the gaps (21.88)-(21.91), among which the last three equations have been illustrated in Chapter 2.

Here we derive (21.88). The Gamma distribution is an exponential family distribution:

whose sufficient statistics is $\varphi (x)={(x,\ln <!--\; FUNCTION\; APPLICATION\; -->x)}^{\text{T}}$ and natural parameter is $𝜃={(-b,a-1)}^{\text{T}}$. Its cumulant function is:

Recall that the expectation of one sufficient statistics is given by the derivative of the cumulant function, therefore:

Finally, according to the defintion $\psi (a)=\frac{{\mathrm{\Gamma }}^{\prime }(a)}{\mathrm{\Gamma }(a)}$, we arrive in:

By using the property of the exponential family, we can get rid of tedious calculus. The rest steps have already been detailed in 21.5.1.6.