Homepage Solution manuals Kevin P. Murphy Machine Learning: a Probabilistic Perspective Exercise 8.5 - Symmetric version of l2 regularized multinomial logistic regression

Exercise 8.5 - Symmetric version of l2 regularized multinomial logistic regression

Answers

We borrow the results from exercise 8.5. Once the l 2 prior is introduced, the likelihood becomes:

l 2 = l c λ 𝐰 c T 𝐰 c .

Therefore (8.126) becomes:

𝐰 c l 2 = i ( y 𝑖𝑐 μ 𝑖𝑐 ) 𝐱 i λ 𝐰 c .

At the unique optimum we have c ,

𝐰 c l 2 = 0 ,

which is identical to:

ŵ c , j = 1 λ i ( y 𝑖𝑐 μ 𝑖𝑐 ) x 𝑖𝑗 .

Therefore:

c ŵ c , j = 1 λ i [ c ( y 𝑖𝑐 μ 𝑖𝑐 ) ] x 𝑖𝑗 ,

whose value is zero since c y 𝑖𝑐 = c μ 𝑖𝑐 = 1 .

User profile picture
2021-03-24 13:42
Comments