Exercise 13.9 - EM for sparse probit regression with Laplace prior

The ordinary Probit regression involves no latent variable. Introducing Laplace prior for the linear weight $\mathbf{\text{𝐰}}$ results in its lasso version. Since Laplace distribution is a continuous mixture of Gaussian according to (13.86), a latent variable ${\tau }^2$ with the same dimension as $\mathbf{\text{𝐰}}$ is introduced. For each component $w_j$ of $\mathbf{\text{𝐰}}$, there is a corresponding latent variable ${\tau }_j^2$ to guide its variance. The PGM for this Probit regression looks like:

The joint distribution is:

where $\mathrm{\Phi }$ is the c.d.f. for a unit Gaussian. According to (13.86):

Hence:

To build the auxiliary function, we assumed $\mathbf{\text{𝐰}}$ as the parameter to be estimated and ${\tau }^2$ as latent variable, thus:

We now extract terms involving $\mathbf{\text{𝐰}}$ from $\log <!--\; FUNCTION\; APPLICATION\; -->p({\tau }^2,\mathbf{\text{𝐰}},\mathbf{\text{𝐲}}|\mathbf{\text{𝐗}},\gamma )$:

Thus we only need to calculate the conditional expectation:

for the E-step. Whose result is already given in exercise 13.8. The M-step is the same as Gaussian-prior Probit regression and hence is omitted.