Exercise 4.16 - Likelihood ratio for Gaussians

Consider a classifier for two classes, the generative distribution for them are two normal distributions $p(x|y=C_i)=𝒩(x|{\mu }_i,{\mathrm{\Sigma }}_i)$, by the Bayes rule:

The first term on r.h.s. is the ratio of likelihood probability.

When we have arbitrary covariance matrices:

As ${\mathrm{\Sigma }}_0,{\mathrm{\Sigma }}_1$ are arbitrary matrices, this formulation cannot be reduced further:

Note that the decision boundary ( $\log <!--\; FUNCTION\; APPLICATION\; -->\frac{p(x|y=1)}{p(x|y=0)}=0$) is a quardratic surface in $D$-dimension space.

When both covariance matrixes are given by $\mathrm{\Sigma }$:

so:

When $\mathrm{\Sigma }$ is a diagonal matrix, we have:

where:

Finally, if $\mathrm{\Sigma }={\sigma }^{2}I$ then:

Note that for the last three cases, a decision boundary is a linear plane in the space, since the quadratic term on $x$ has been canceled in $\mathrm{\Phi }$.