Another service from Omega

Example of Bayes' Theorem


*****

Problem

A box contains three coins. The first coin is fair, the second is two-headed and the third has probability of heads equal to 3/4. One of these coins is selected at random from the box and tossed. What is the probability that the coin is two-headed if the coin landed on heads?

Solution


This is a typical elementary application of Bayes' Theorem.

Recall that Bayes' theorem is nothing but the commutativity of conjunction (AND) applied to the product rule of probability i.e.

P(H and D | O) = P(D and H | O)

using the product rule on both sides, we obtain

P(H |DO) P(D|O) = P(D|HO) P(H|O)

and dividing through by P(D|O) we obtain the celebrated (hurray!)

> Bayes' rule:

                             P(D|HO) P(H|O)
                 P(H|DO) :=  --------------
                                 P(D|O)

Notice that the denominator P(D|O) (known as the evidence) is independent of the hypothesis H and it can be thought as a proportionality constant that makes the total probability of mutually exclusive and exhaustive hypothesis to be equal to one. For this reason, Bayes' rule is usually written as the left hand side proportional to the numerator of the right hand side or,

(Posterior) propTo (Likelihood)x(Prior)


Let us denote by H1, H2 and H3 the three mutually exclusive and exhaustive hypotheses corresponding to "First coin", "Second coin", and "Third coin" being the one chosen.

Let D be denote the fact that we observed the result of one toss to be heads and let O denote the prior information (a large conjunction of facts) that is necessary to understand the statement of this problem. With this notation the answer to the question: What's the probability that the coin is two-headed if we observe a head as the result of a single flip, is given by P(H2|DO). Applying Bayes' rule we get

P(H2|DO) propTo P(D|H2O) P(H2|O) = 1 x 1/3


Note: if you think that H2O above has anything to do with water.... you are flanking this course.

The likelihood is 1 in this case since a two-headed coin always lands on heads. The prior is 1/3 since according to the information given in the problem (which is part of O) the coin is chosen at random from the box of three coins. To compute the proportionality constant (i.e. the evidence P(D|O)) we need to repeat the last equation for H1 and H3. We have,

P(H1|DO) propTo 1/2 x 1/3


the likelihood is 1/2 for H1 since a fair coin has equal chance of landing heads or tails. Finally,

P(H3|DO) propTo 3/4 x 1/3


in this case the likelihood is 3/4 since according to O, the funny coin has chance 3/4 of showing heads.

Now, from the fact that H1,H2 and H3 are mutually exclusive and exhaustive hypotheses, we can write:

P(H1|DO)+P(H2|DO)+P(H3|DO) = 1


from where we can compute the evidence:

P(D|O) = 1/2 x 1/3 + 1 x 1/3 + 3/4 x 1/3 = 3/4


Hence, the final answer is:

P(H2|DO) = (1/3) / (3/4) = 4/9 or about 44%


Link to the commands in this file
Carlos Rodriguez <carlos@math.albany.edu>
Last modified: Thu Sep 10 10:19:14 EDT 1998