Exercise 13.7 - Prior for the Bernoulli rate parameter in the spike and slab model

Answers

Recall that the prior takes the following decomposition:

p (γ | α_{1}, α_{2}) = \prod_{d = 1}^{D} p (γ_{d} | α_{1}, α_{2}) = \prod_{d = 1}^{D} p (γ_{d} | π_{d}) \cdot Beta (π_{d} | α_{1}, α_{2}) .

We now integrate out $π_{d}$ for the $d$ -th parameter:

\begin{align} p (γ_{d} | α_{1}, α_{2}) = & \frac{1}{B (α_{1}, α_{2})} \int π_{d}^{γ_{d}} {(1 - π_{d})}^{(1 - γ_{d})} π_{d}^{α_{1} - 1} {(1 - π_{d})}^{α_{2} - 1} d π_{d} \\ = & \frac{1}{B (α_{1}, α_{2})} \int π_{d}^{α_{1} + γ_{d} - 1} {(1 - π_{d})}^{α_{2} + 1 - γ_{d} - 1} d π_{d} \\ = & \frac{B (α_{1} + γ_{d}, α_{2} + 1 - γ_{d})}{B (α_{1}, α_{2})} = \frac{Γ (α_{1} + α_{2})}{Γ (α_{1}) Γ (α_{2})} \cdot \frac{Γ (α_{1} + γ_{d}) Γ (α_{2} + 1 - γ_{d})}{Γ (α_{1} + α_{2} + 1)} . \end{align}

Therefore( $N_{1}$ marks the number of activa parameters in $γ$ ):

\begin{align} p (γ | α_{1}, α_{2}) = & (\binom{N}{N_{1}}) \cdot \frac{Γ {(α_{1} + α_{2})}^{N}}{Γ {(α_{1})}^{N} Γ {(α_{2})}^{N}} \cdot \frac{Γ {(α_{1} + 1)}^{N_{1}} Γ {(α_{2} + 1)}^{N - N_{1}}}{Γ {(α_{1} + α_{2} + 1)}^{N}} \\ = & (\binom{N}{N_{1}}) \cdot \frac{α_{1}^{N_{1}} α_{2}^{N - N_{1}}}{{(α_{1} + α_{2})}^{N}} . \end{align}

Hence $p (γ)$ is this case is a binomial distribution.

solour_lfq

2021-03-24 13:42