Exercise 3.11 - Bayesian analysis of the exponential distribution

Answers

The exponential distribution is also crucial for the queueing theory. The log-likelihood for an exponential distribution with density:

p (x | 𝜃) = 𝜃 \cdot \exp (- 𝜃 \cdot x)

is:

\ln p (𝒟 | 𝜃) = N \cdot \ln 𝜃 - 𝜃 \cdot \sum_{n = 1}^{N} x_{n},

whose derivative is:

\frac{\partial}{∂𝜃} \ln p (𝒟 | 𝜃) = \frac{N}{𝜃} - \sum_{n = 1}^{N} x_{n}

Thus for question (a), we have:

𝜃_{MLE} = \frac{N}{\sum_{n = 1}^{N} x_{n}} .

For question (b), $𝜃_{MLE} = 5$ .

For question (c), we begin with an exponential prior distribution:

p (𝜃 | λ) = λ \cdot \exp (- λ \cdot 𝜃),

whose expectation is:

\int_{0}^{\infty} λ \cdot 𝜃 \cdot \exp (- λ \cdot 𝜃) d 𝜃 .

Integration by parts (or resort to the nomarlization term of the Gamma distribution) yields:

𝔼 (𝜃) = \frac{1}{λ} .

So $\hat{λ} = 3 .$

For question (d), the posterior distribution is:

\begin{aligned} p (𝜃 | 𝒟, λ) & = \frac{p (𝒟 | 𝜃) \cdot p (𝜃 | λ)}{p (𝒟 | λ)} \\ = \frac{1}{p (𝒟 | λ)} \cdot 𝜃^{N} \cdot \exp (- 𝜃 \cdot \sum_{n = 1}^{N} x_{n}) \cdot λ \cdot \exp (- λ \cdot 𝜃) \\ = \frac{𝜃^{N} \cdot λ}{p (𝒟 | λ)} \cdot \exp (- 𝜃 \cdot (λ + \sum_{n = 1}^{N} x_{n})) . \end{aligned}

Hence the posterior is a Gamma distribution with hyperparameters:

a = N + 1,

b = λ + \sum_{n = 1}^{N} x_{n} .

The evidence is given by: $\frac{λ \cdot Γ (a)}{b^{a}}$ , a function of $λ$ and $𝒟$ . Hence the exponential distribution is not the conjugate distribution of itself, answering question (e).

For question (f), the posterior mean is the mean of the Gamma distribution:

\frac{a}{b} = \frac{N + 1}{λ + \sum_{n = 1}^{N} x_{n}} .

Compared with the MLE, the posterior mean has additional terms for both the numerator and the denominator as basic knowledge when $N$ is relatively small. The influence of using this prior is tantamount to introducing a prior sample with value $λ$ .

solour_lfq

2021-03-24 13:42

Exercise 3.11 - Bayesian analysis of the exponential distribution

Answers

Comments

Add answer