Exercise 7.4 - MLE for $\sigma^{2}$ for linear regression

Answers

The likelihood is given by:

\begin{align} p (𝒟 | 𝐰, σ^{2}) = & p (𝐘 | 𝐰, σ^{2}, 𝐗) \\ = & \prod_{n = 1}^{N} p (y_{n} | 𝐱_{n}, 𝐰, σ^{2}) \\ = & \prod_{n = 1}^{N} 𝒩 (y_{n} | 𝐰^{T} 𝐱_{n}, σ^{2}) \\ = & \frac{1}{{(2 π σ^{2})}^{\frac{N}{2}}} \cdot \exp {- \frac{1}{2 σ^{2}} \sum_{n = 1}^{N} {(y_{n} - 𝐰^{T} 𝐱_{n})}^{2}} . \end{align}

As for $σ^{2}$ :

\frac{\partial}{\partial σ^{2}} \log p (𝒟 | 𝐰, σ^{2}) = - \frac{N}{2 σ^{2}} + \frac{1}{2 {(σ^{2})}^{2}} \sum_{n = 1}^{N} {(y_{n} - 𝐰^{T} 𝐱_{n})}^{2} .

Setting it to zero yields:

σ_{MLE}^{2} = \frac{1}{N} \sum_{n = 1}^{N} {(y_{n} - 𝐰^{T} 𝐱_{n})}^{2} .

solour_lfq

2021-03-24 13:42