Exercise 3.7 - Bayesian analysis of the Poisson distribution

Answers

The conjugate prior for the Poisson distribution is the Gamma distribution:

Ga ( λ | a , b ) = b a Γ ( a ) λ a 1 exp ( λ b ) .

The posterior for a Bayesian Poisson model reads:

p ( λ | 𝒟 , a , b ) = p ( 𝒟 | λ ) p ( λ | a , b ) p ( 𝒟 | a , b ) = b a n = 1 N x n ! p ( 𝒟 | a , b ) Γ ( a ) λ a + n = 1 N x n 1 exp ( λ ( N + b ) ) = Ga ( a + n = 1 N x n , N + b ) ,

in which the last step follows the normalization condition. We now have the evidence:

p ( 𝒟 | a , b ) = b a Γ ( a + n = 1 N x n ) n = 1 N x n ! ( N + b ) a + n = 1 N x n Γ ( a ) .

Finally, be a and b approximate zero, the posterior mean approaches n = 1 N x n N , the same as the MLE, i.e., a = 0 , b = 0 is a non-informative prior. One should note that this property does not hold for all Bayesian analysis, setting all hyperparameters to zero does not necessarily gracefully degenerate the posterior mean to the MLE. Since the names and definitions of those symbols might differ.

User profile picture
2021-03-24 13:42
Comments