4.3 Multiple Hypotheses
We start this section with an example.
Example 1 What is the probability that you are holding one of either the High or the Low Deck having drawn five 9's in a row from that deck?
Show solution
We have observed the following data:
d
a
t
a
≡
{
``We've drawn one card, and it is a 9, replaced and reshuffled, redrawn and observed another 9, repeated this procedure and observed three more 9's, for a total of five 9's in a row.''
Technically, drawing 5 9's in a row should give us really strong confidence that you are drawing from the High Deck, because we would have
Specify the prior probabilities for the models being consideredP ( H ) = 0.5 P ( L ) = 0.5
Write the top of Bayes' Rule for all models being consideredP ( H | d a t a = 5 9's in a row ) ∼ P ( d a t a = 5 9's in a row | H ) P ( H ) P ( L | d a t a = 5 9's in a row ) ∼ P ( d a t a = 5 9's in a row | L ) P ( L )
Put in the likelihood and prior valuesP ( H | d a t a = 5 9's in a row ) ∼ 9 55 × 9 55 ⋯ 9 55 ⏟ 5 times × P ( H ) ∼ ( 9 55 ) 5 × 0.5 = 0.0000587 P ( L | d a t a = 5 9's in a row ) ∼ ( 2 55 ) 5 × 0.5 = 0.0000000318
Add these values for all modelsK = 0.0000587 + 0.0000000318 = 0.0000587318
Divide each of the values by this sum, K P ( H | d a t a = 5 9's in a row ) = 0.0000587 0.0000587318 = 0.99946 P ( L | d a t a = 5 9's in a row ) = 0.0000000318 0.0000587318 = 0.00054
which is fantastically on the side of the high deck, even though we might start getting suspicious in this situation.
Example 2 What is the probability that you are holding one of either the High or the Low Deck having drawn m m m = 1 , 2 , 3 , ⋯
Show solution
In general, if we look at m m
Specify the prior probabilities for the models being consideredP ( H ) = 0.5 P ( L ) = 0.5
Write the top of Bayes' Rule for all models being consideredP ( H | d a t a = m 9's in a row ) ∼ P ( d a t a = m 9's in a row | H ) P ( H ) P ( L | d a t a = m 9's in a row ) ∼ P ( d a t a = m 9's in a row | L ) P ( L )
Put in the likelihood and prior valuesP ( H | d a t a = m 9's in a row ) ∼ 9 55 × 9 55 ⋯ 9 55 ⏟ m times × P ( H ) ∼ ( 9 55 ) m × 0.5 P ( L | d a t a = m 9's in a row ) ∼ ( 2 55 ) m × 0.5
Add these values for all modelsK = ( 9 55 ) m × 0.5 + ( 2 55 ) m × 0.5
Divide each of the values by this sum, K It is clear from Table 4.1 that after drawing five 9's using our procedure, it should be extraordinarily likely that we are holding the High Deck.
However, after a certain number of 9's observed, something starts to bother us. Perhaps not after five 9's, but what if the procedure were repeated and we drew ten 9's in a row? Or perhaps twenty 9's. At some point, we'd refuse to believe this is the High Deck because, although it was true that there are more 9's in the High Deck, there are many more other cards in the High Deck that we should see. What do we do in this case?
Example 3 What is the probability that you are holding one of either the High, Low, or Nines Deck having drawn m
Show solution
The proper thing to do is to introduce a new model, say, a Nines deck. Clearly this model should have a very low prior probability, because we didn't even consider it before we saw the streak of 9's. Let's say that we assign the prior probability for the Nines deck to be a one in a million. To make all of the prior probabilities add up to 1, then the prior probabilities for the High and Low Deck must be a little less than 0.5. After that, we simply apply the Bayes' Recipe as before
Note
What is interesting here is that once we admit that there are many possible models we could consider, we realize that we have these models in our head all the time, or we construct them as we need them. Every model comparison is a multiple model comparison, with most of the models with very low prior probabilities that our brain naturally suppresses until needed. Mathematically, we need to unsuppress them as needed.
Specify the prior probabilities for the models being consideredP ( N ) = 1 1,000,000 = 0.000001 P ( H ) = 0.4999995 P ( L ) = 0.4999995
Write the top of Bayes' Rule for all models being consideredP ( N | d a t a = m 9's in a row ) ∼ P ( d a t a = m 9's in a row | N ) P ( N ) P ( H | d a t a = m 9's in a row ) ∼ P ( d a t a = m 9's in a row | H ) P ( H ) P ( L | d a t a = m 9's in a row ) ∼ P ( d a t a = m 9's in a row | L ) P ( L )
Put in the likelihood and prior valuesP ( N | d a t a = m 9's in a row ) ∼ 1 × P ( N ) = 0.000001 P ( H | d a t a = m 9's in a row ) ∼ 9 55 × 9 55 ⋯ 9 55 ⏟ m times × P ( H ) ∼ ( 9 55 ) m × 0.4999995 P ( L | d a t a = m 9's in a row ) ∼ ( 2 55 ) m × 0.0 .4999995
Add these values for all modelsK = 0.000001 + ( 9 55 ) m × 0.4999995 + ( 2 55 ) m × 0.4999995
Divide each of the values by this sum, K Figure 4.3. Drawing a number of 9's in a row, possibly from a High, Low, and Nines deck.We have a clear picture here in Figure 4.3. As we initially draw 9's, our confidence that we're holding the High Deck goes up, at the expense of our confidence that we're holding the Low Deck. At a certain point (around six 9's in our example), our confidence in the High Deck starts to drop and we become more confident that something odd is happening, and our previously ignored model of the Nines deck becomes more likely. Eventually, this new model is the one in which we are the most confident.
Imagine further that if, after drawing ten 9's in a row we draw a 1. What do we do then? The likelihood for the Nines deck goes to zero instantly - the probability of drawing a 1 from a Nines deck is zero, P ( 1 | N ) = 0
Note
The creative part of science is not in the calculations performed, but in the generation of new and useful models. Until we come up with a better model for our data we make do with the ones that we have, all the while being aware that a better model may come into play later. Newton's Theory of Gravity was used for over 200 years, even when there was known data that made it less likely, until it was replaced by Einstein's Theory of Gravity. Newton's Laws, however, are still used in nearly all gravitational calculations because it is “good enough” and is a lot easier to work with practically.
Your Turn
Drawing a 3
Complete the example demonstrating the updated probabilities for the High and Low Deck, having drawn a 9, 7, and a 3. Compare with the case of drawing just the 9 and the 7, and discuss how it matches your intuition.
Your Turn
Nines and an Eight
Repeat the analysis of the sequence of 9's drawn in a row with an added hypothesis of a deck with one hundred 9's and one 8. Discuss the results. Demonstrate what happens to the probabilities for all of the hypotheses after drawing one 8, after ten 9's in a row. Discuss.
Your Turn
Weird Coins
I tell you that I have a coin that could have both sides heads, both sides tails, or a normal single-heads single-tails coin.
Before seeing the data, what would be a reasonable prior probability for the three hypotheses H 0 H 1 H 2 Would this have been different if you had simply been given a coin by a friend to flip to see who has to do the dishes? Why or why not? Now I flip the coin once, and get a heads. Write down the likelihood of this data given each of the models. In other words, what are the values of:P ( data=1 heads | H 0 ) P ( data=1 heads | H 1 ) P ( data=1 heads | H 2 ) Apply Bayes' Recipe, and determine the probability of each of these three models given this data. In other words, what are the values of:P ( H 0 | data=1 heads ) P ( H 1 | data=1 heads ) P ( H 2 | data=1 heads ) Apply this recipe for the case of observing 3 heads in a row. Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0 .
