Exercise 24.7 - Gibbs sampling for logistic regression with the Student approximation

Answers

The complete likelihood with (24.88)-(24.91) is:

\begin{aligned} p (λ, 𝐙, 𝐰, 𝐘 | 𝐗, v) & = p (λ | v) \cdot p (𝐰) \cdot p (𝐙 | 𝐗, 𝐰, λ) \cdot p (𝐘 | 𝐙) \\ = \prod_{n = 1}^{N} Ga (λ_{n} | \frac{v}{2}, \frac{v}{2}) \\ \cdot p (𝐰) \cdot \prod_{n = 1}^{N} 𝒩 (z_{n} | 𝐰^{T} 𝐱_{n}, λ_{n}^{- 1}) \\ \cdot \prod_{n = 1}^{N} 𝕀 [y_{n} \sim z_{n}] . \end{aligned}

For $λ$ , we have:

p (λ_{n} | \dots) \propto λ_{n}^{\frac{v}{2} - 1} \cdot \exp {- \frac{v λ_{n}}{2}} \cdot λ_{n}^{\frac{1}{2}} \cdot \exp {- \frac{λ_{n} {(z_{n} - 𝐰^{T} 𝐱_{n})}^{2}}{2}} .

So the conditional posterior is a Gamma distribution with parameters:

\frac{v + 1}{2},

\frac{v}{2} + \frac{{(z_{n} - 𝐰^{T} 𝐱_{n})}^{2}}{2} .

For $𝐙$ , we have:

p (z_{n} | \dots) \propto \exp {- \frac{λ_{n} {(z_{n} - 𝐰^{T} 𝐱_{n})}^{2}}{2}},

s.t. $y_{n} \sim z_{n}$ , therefore $z_{n}$ follows a truncated Gaussian.

Finally, $𝐰$ follows a Gaussian, as in exercise 24.5 and 24.6.

solour_lfq

2021-03-24 13:42