Exercise 11.14 - EM for censored linear regression

The model for censored linear regression (non-Bayesian version) is:

The observed variables are $y_i$, $c_i$ and ${\mathbf{\text{𝐱}}}_i$ and we are to estimate $\mathbf{\text{𝐰}}$ and ${\sigma }^2$. The latent variable in this model is $z_i$. The complete likelihood is:

if $y_i\ne c_i$, and is:

if $y_i=c_i$, which implies $z_i\ge c_i$. We observe that the integral is going to appear inside the logarithm operator, so it is better to approximate this value from its moments. One possible approximation is to use (11.137) and (11.138), so when $y_i=c_i$, the first and the second moment of $z_i$ are:

respectively. Note that this is a variant version of the E-step.

The log likelihood for the entire dataset now becomes:

where ${\sigma {}_{\prime }^{}}^i.$

Finally, in the M-step, we take the partial gradient of the log likelihood w.r.t. $\mathbf{\text{𝐰}}$ and ${\sigma }^2$, set them to zero in order to update them. This step is straightforward given $H$’s gradient can be automatically solved efficiently.