Bayesian classification

Let x be an observation and C(x) a Bernoulli random variable, function of x. We are interested in computing the probability distribution of C, conditioned on the value of x.

P(C | x) * P(x) = P(C and x) = P(x | C) * P(C)

• P(C): prior probability
• P(x | C): likelihood of x assuming C
• P(x): evidence
• P(C | x): posterior likelihood of C given x

Simple Bayes' classifier

Select C such that P(C | x) is maximum.

Risk

When we make a misclassification we incur in a cost. Risk: measure of the uncertainty of this loss.

• Expected loss
• VaR
• Worst Conditional Expectation

Loss:

l(i, k) = Loss incurred in misclassifying an instance of class i as an instance of class k

Expected risk:

R(C_i | x) = SUM(k = 1, K) l(i,k) P(C_k | x)