Homepage › Solution manuals › Kevin P. Murphy › Machine Learning: a Probabilistic Perspective › Exercise 19.1 - Derivation of the log partition function

Machine Learning: a Probabilistic Perspective

Exercise 19.1 - Derivation of the log partition function

Answers

Recall the definition of the partition function:

Z (𝜃) = \sum_{𝐲} \prod_{c \in 𝒞} ψ_{c} (𝐲_{c} | 𝜃_{c}),

where $𝒞$ is the collection of all maximal cliques. The rest is straightforward algebra:

\begin{align} \frac{\partial \log Z (𝜃)}{\partial 𝜃_{c^{'}}} = & \frac{\partial}{\partial 𝜃_{c^{'}}} \log \sum_{𝐲} \prod_{c \in 𝒞} ψ_{c} (𝐲_{c} | 𝜃_{c}) \\ = & \frac{1}{Z (𝜃)} \sum_{𝐲} \frac{\partial}{\partial 𝜃_{c^{'}}} \prod_{c \in 𝒞} ψ_{c} (𝐲_{c} | 𝜃_{c}) \\ = & \frac{1}{Z (𝜃)} \sum_{𝐲} (\prod_{c \in 𝒞, c \neq c^{'}} ψ_{c} (𝐲_{c} | 𝜃_{c})) \cdot \frac{\partial}{\partial 𝜃_{c^{'}}} ψ_{c^{'}} (𝐲_{c^{'}} | 𝜃_{c^{'}}) \\ = & \frac{1}{Z (𝜃)} \sum_{𝐲} (\prod_{c \in 𝒞, c \neq c^{'}} ψ_{c} (𝐲_{c} | 𝜃_{c})) \cdot \frac{\partial}{\partial 𝜃_{c^{'}}} \exp {𝜃_{c^{'}}^{T} ϕ_{c^{'}} (𝐲_{c^{'}})} \\ = & \frac{1}{Z (𝜃)} \sum_{𝐲} (\prod_{c \in 𝒞} ψ_{c} (𝐲_{c} | 𝜃_{c})) \cdot ϕ_{c^{'}} (𝐲_{c^{'}}) \\ = & \sum_{𝐲} ϕ_{c^{'}} (𝐲_{c^{'}}) \cdot (\frac{1}{Z (𝜃)} \prod_{c \in C} ψ_{c} (𝐲_{c} | 𝜃)) \\ = & \sum_{𝐲} ϕ_{c^{'}} (𝐲_{c^{'}}) \cdot p (𝐲 | 𝜃) \\ = & 𝔼 [ϕ_{c^{'}} (𝐲_{c^{'}}) | 𝜃] . \end{align}

solour_lfq

2021-03-24 13:42

Comments