Exercise 17.2 - Two filter approach to smoothing in HMMs

Answers

For r t ( i ) = p ( z t = i | 𝐱 t + 1 : T ) , we have:

p ( z t = i | 𝐱 t + 1 : T ) = ∑ j p ( z t = i , z t + 1 = j | 𝐱 t + 1 : T ) = ∑ j p ( z t + 1 = j | 𝐱 t + 1 : T ) ⋅ p ( z t = i | z t + 1 = j , 𝐱 t + 1 : T ) = ∑ j p ( z t + 1 = j | 𝐱 t + 1 : T ) ⋅ p ( z t = i | z t + 1 = j ) = ∑ j p ( z t + 1 = j | 𝐱 t + 1 : T ) ⋅ Ψ − ( j , i ) ,

where Ψ − denotes the transform matrix in an inverse sense. In the third step we make use of the conditional independence property within a HMM.

We further have:

p ( z t + 1 = j | 𝐱 t + 1 : T ) = p ( z t + 1 = j | 𝐱 t + 1 , 𝐱 t + 2 : T ) ∝ p ( z t + 1 = j , x t + 1 , x t + 2 : T ) ∝ p ( z t + 1 = j | x t + 2 : T ) ⋅ p ( x t + 1 | z t + 1 = j , x t + 2 : T ) = r t + 1 ( j ) ⋅ p ( x t + 1 | z t + 1 = j ) = r t + 1 ( j ) ⋅ ψ t + 1 ( j ) .

Therefore we can calculate r t ( i ) recursively:

r t ( i ) ∝ ∑ j r t + 1 ( j ) ⋅ ψ t + 1 ( j ) ⋅ Ψ − ( j , i ) .

Finally, it is straightforward to rewrite γ t ( i ) in terms of new factors:

γ t ( i ) = p ( z t = i | 𝐱 1 : T ) ∝ p ( z t = i , 𝐱 1 : T ) = p ( z t = i ) ⋅ p ( 𝐱 1 : T | z t = i ) = p ( z t = i ) ⋅ p ( 𝐱 1 : t | z t = i ) ⋅ p ( 𝐱 t + 1 : T | z t = i , 𝐱 1 : t ) = p ( z t = i ) ⋅ p ( 𝐱 1 : t | z t = i ) ⋅ p ( 𝐱 t + 1 : T | z t = i ) = 1 p ( z t = i ) ⋅ p ( 𝐱 1 : t , z t = i ) ⋅ p ( 𝐱 t + 1 : T , z t = i ) ∝ 1 p ( z t = i ) ⋅ p ( z t = i | 𝐱 1 : t ) ⋅ p ( z t = i | 𝐱 t + 1 : T ) = α t ( i ) ⋅ r t ( i ) ∏ t ( i ) .

This finishes the deduction. Note that in all deduction in HMM we can freely use ∝ to drop out the evidence and make use of the graphical conditional independence.

User profile picture
2021-03-24 13:42
Comments