Bayes Theorem

Conditional Probability and some useful formulas

The conditional probability of B given A is:

\(P(A|B)=\frac{P(A\cap B)}{P(B)}\)

thus

\(P(A\cap B)=P(A|B)*P(B)\)

and the bayes’rule:

\(P(B|A)\ =\ \frac{P(A\cap B)}{P(A)}\ =\ \frac{P(A|B)*P(B)}{P(A)}\)

Prior probability, posterior probability and likelihood

Consider we are making dinner and we are going to cook a fish. There are different approaches to cook the fish: steam and pan-sear. As a result of cooking, there may be two different outcomes: the final dish may be delicious or horrible (different individual varies in different cooking skills!).

Below are the likelihood given for different conditions.

\(L(Outcome|Approach)\) = \(P(Outcome|Approach)\)

Approach	Outcome
steam	70% delicious, 30% horrible
pan-sear	80% delicious, 20% horrible

And in general, let us assume that the chances of outcomes are 50/50, which means the Prior probabilities are:

\(P(delicious)\ =\ 0.5\) and \(P(horrible)\ =\ 0.5\)

Now the guest Bob is served with a poorly cooked fish (horrible outcome), what is the probability of that Alice pan-seared the fish? The probability we are looking for is

\(P(pan-sear|horrible)\).

This is the posterior probability - given the result, we try to find the reasons (or approaches) that caused this result. According to Bayes’ rule,

\(P(pan-sear|horrible)\) = \(\frac{P(horrible|pan-sear)}{P(horrible)}\) = \(\frac{0.2}{0.5}\) = \(0.4\)

So, Bob knows that there are a 40% chance that Alice pan-seared the horrible fish.