3.2 Coin Flipping

We'll start with some simple examples of coin flipping, asking some simple questions, and move to more complex observations and unintuitive conclusions.

Example 1 What is the probability of flipping three heads in a row, with a fair coin?

Show solution

We can approach this problem in two different ways. The first way, is a brute-force counting method with the definition of probability for exclusive events (using Equation 1.2) and the second way makes use of the other rules of probability. In the first way, we simply outline every possible combination of three flips, see how many are “three heads in a row”, as we show in the margin.

Because there is only one case of “H H H” in all eight, the probability of three heads in a row is

$\begin{eqnarray*} P\left(\text{three heads in a row}\right)=1/8 \end{eqnarray*}$

which is an unlikely outcome, but not extremely so (see Table 1.1 on page 51).

In terms of the rule of probability, we have

$\begin{eqnarray*} P\left(\text{three heads in a row}\right)=P(H_{1} \text{\bf\ and } H_{2} \text{\bf\ and } H_{3}) \end{eqnarray*}$

where $H_{1}$ is heads on the first flip, $H_{2}$ is heads on the second flip, etc... Because these are independent events (Section 1.5), the probability is just the product of the probabilities of the individual events (Equation 1.9)

$\begin{eqnarray*} P\left(\text{three heads in a row}\right)&=&P(H_{1} \text{\bf\ and } H_{2} \text{\bf\ and } H_{3})\\ &=&P(H_{1})\times P(H_{2})\times P(H_{3})\\ &=&\frac{1}{2}\times\frac{1}{2}\times\frac{1}{2}\\ &=&\frac{1}{8} \end{eqnarray*}$

the same answer as before.

Example 2 What is the probability of flipping thirty heads in a row, with a fair coin?

Show solution

Our intuition will clearly insist that this will be a very small number, but how small? Our first method, of listing all of the possibilities gets quite a bit cumbersome with this question. The second method is quite straightforward

$\begin{eqnarray*} P\left(\text{thirty heads in a row}\right)&=&P(H_{1} \text{\bf\ and } H_{2} \text{\bf\ and } \cdots \text{\bf\ and } H_{30})\\ &=&P(H_{1})\times P(H_{2})\times \cdots \times P(H_{30})\\ &=&\overbrace{\frac{1}{2}\times\frac{1}{2}\times\cdots \times \frac{1}{2}}^{\text{30 times}}\\ &=&\left(\frac{1}{2}\right)^{30}\\ &=&0.000000001 \text{ (one in a billion!)} \end{eqnarray*}$

This is virtually impossible (Table 1.1).

Example 3 What is the probability of flipping two heads in three flips, with a fair coin?

Show solution

Our intuition suggests that this should be a reasonably common occurrence. We address this problem in exactly the same two ways: first, by counting, the second with the rules of probability. In the first method, we observe from the table that there are three ways of getting two heads: “T H H,” “H T H,” and “H H T.” Thus,

$\begin{eqnarray*} P\left(\text{two heads in three flips}\right) = \frac{3}{8} \end{eqnarray*}$

In the second method we write

$\begin{eqnarray*} \lefteqn{P\left(\text{two heads in three flips}\right)=}\\ && P\left( (T_{1} \text{\bf\ and } H_{2} \text{\bf\ and } H_{3}) \text{\bf\ or } (H_{1} \text{\bf\ and } T_{2} \text{\bf\ and } H_{3}) \text{\bf\ or } (H_{1} \text{\bf\ and } H_{2} \text{\bf\ and } T_{3}) \right) \end{eqnarray*}$

from which we can apply the sum rule for exclusive events (Equation 1.11) and, like before, the product rule for independent events (Equation 1.9),

$\begin{eqnarray*} \lefteqn{P\left(\text{two heads in three flips}\right)=}\\ && P(T_{1} \text{\bf\ and } H_{2} \text{\bf\ and } H_{3})+P(H_{1} \text{\bf\ and } T_{2} \text{\bf\ and } H_{3})+P(H_{1} \text{\bf\ and } H_{2} \text{\bf\ and } T_{3})\\ &=&\left(\frac{1}{2}\times\frac{1}{2}\times\frac{1}{2}\right)+\left(\frac{1}{2}\times\frac{1}{2}\times\frac{1}{2}\right)+\left(\frac{1}{2}\times\frac{1}{2}\times\frac{1}{2}\right)\\ &=&\frac{1}{8}+\frac{1}{8}+\frac{1}{8}=\frac{3}{8} \end{eqnarray*}$

which is about a 38% chance, slightly unlikely (Table 1.1).

Example 4 What is the probability of flipping ten heads in thirty flips, with a fair coin?

Show solution

Once the numbers start getting large, our intuition fails, and we can't list all the possibilities. In order to proceed, we need to develop a systematic way of approaching these sorts of problems. Essentially it comes down to two parts:

What is the probability of one particular sequence being considered?
How many ways can this type of sequence appear in the process described in the question?

Point 1 is asking, what is the probability of this particular sequence:

H H H H H H H H H H T T T T T T T T T T T T T T T T T T T T

or this sequence:

T T H T T T T H H H H T T T T T H T T H T T T T T T H H T H

Although it is unintuitive, mathematically both of these specific sequences have exactly the same probability: each head or tail has equal probability, is not related to the others, and there are the same number of them. So we have

$\begin{eqnarray*} \lefteqn{P\left(\text{HHHHHHHHHHTTTTTTTTTTTTTTTTTTTT}\right)=}\\ &&P\left(\text{TTHTTTTHHHHTTTTTHTTHTTTTTTHHTH}\right)\\ &=&\left(\frac{1}{2}\right)^{30}\\ &=&0.000000001 \text{ (one in a billion!)} \end{eqnarray*}$

Every single specific length-thirty sequence of heads and tails has the same probability, one in a billion.

Point 2 is asking, how many sequences are there of thirty heads and tails where ten of them are heads? Another way of phrasing it is, given a sequence like:

H H H H H H H H H H T T T T T T T T T T T T T T T T T T T T

how many different ways can I rearrange this sequence and get a unique sequence?

Counting the Rearrangements

We are going to determine the answer to our question in small steps.

First, we ask,

Example 5 How many ways can we rearrange the unique symbols A, B, C, and D?

Show solution

To make this intuitive, we set up four empty boxes and we imagine placing our symbols in the boxes, one at a time. How many choices do we have? For the first box, we have four choices. For each of these choices, we've removed one of the symbols, and one of the boxes. Thus, we are left with three remaining symbols for each choice, and three remaining boxes. For each of the original four choices, we now have three choices for the second box. This immediately leads to $4\times 3=12$ possibilities by the time we've filled two boxes. For each of these twelve possibilities, there are two symbols remaining and two boxes. Continuing this logic, we have two choices for the third box, and then only one choice for the final box. In summary, for each of the four choices for the first box we have three choices for the second, two choices for the third, and one for the final box. Thus we have

Note

Choices	Remaining Symbols
A B □ □	C D
A C □ □	B D
A D □ □	B C
B A □ □	C D
B C □ □	A D
B D □ □	A C
C A □ □	B D
C B □ □	A D
C D □ □	A B
D A □ □	B C
D B □ □	A C
D C □ □	A B

$\begin{eqnarray*} \left(\text{number of rearrangements of four different symbols}\right)=4\times 3 \times 2 \times 1 = 24 \end{eqnarray*}$

In general we have

$\begin{eqnarray} \nonumber C(N)&=&N\times (N-1) \times \cdots \times 2 \times 1 \\ &=&N! \label{eq:rearrangements_unique} \end{eqnarray}$

Number of Rearrangements of $N$ Unique Symbols where we've introduced the notation for the factorial of $N$ as $N!$ .

Note

Number of Rearrangements of $N$ Unique Symbols

$\begin{eqnarray*} C(N)&=&N\times (N-1) \times \cdots \times 2 \times 1 \\ &=&N! \end{eqnarray*}$

Example 6 How many ways can we rearrange the symbols A, A, A, and D?

Show solution

By eye we can see that there are only four rearrangements of these symbols. How is this different from the previous question with four symbols? We can imagine going from the first question, with four unique symbols “A B C D,” and replace both “B” and “C” with “A” to get it. “BC” and “CB” are different sequences of unique symbols. However, if we replace “B” with an “A” and “C” with an “A”, both sequences become the same sequence, namely “AA”. If we try to blindly apply Equation 3.1, the one for the number of rearrangements of unique symbols, to the case where there are duplicates, we will overestimate the number of rearrangements - we are over counting duplicate subsequences. Further, we can be specific about how much we are over counting and thus find a new equation which includes the possibility of duplicates.

For example, if we have three duplicates in a sequence, the number of over countings will be the number of possible rearrangements of three unique symbols, because all of these rearrangements result in the same sequence of duplicate symbols. Thus, our procedure should be,

$\begin{eqnarray*} \left(\text{number of rearrangements of ``A A A D''}\right)&=&\frac{\left(\text{number of rearrangements of four \emph{unique} symbols}\right)}{\left(\text{number of rearrangements of the over-counted duplicate three symbols}\right)}\\ &=&\frac{4!}{3!} \\ &=&\frac{4\times 3 \times 2 \times 1}{3 \times 2 \times 1}\\ &=&4 \end{eqnarray*}$

Example 7 How many ways are there of rearranging the symbols “A A A D D”?

Show solution

Following the same logic, we have

$\begin{eqnarray*} \overbrace{\underbrace{\text{A A A }}_{\text{$3!$ ways of rearranging 3 duplicates}}\underbrace{\text{D D}}_{\text{$2!$ ways of rearranging 2 duplicates}}}^{\text{$5!$ ways of rearranging 5 \emph{unique} symbols}} \end{eqnarray*}$

$\begin{eqnarray*} \left(\text{number of rearrangements of ``A A A D D''}\right)&=&\frac{5!}{3!2!} \\ &=&\frac{5\times 4 \times 3 \times 2 \times 1}{(3 \times 2 \times 1) \times (2 \times 1)}\\ &=&\frac{120}{6\times 2} \\ &=&10 \end{eqnarray*}$

Note

All possible results of rearranging the symbols “A A A D D”:

1	A A D D A
2	D A A D A
3	A D A D A
4	D A A A D
5	D A D A A
6	A A D A D
7	D D A A A
8	A D D A A
9	A A A D D
10	A D A A D

Sequences of Heads and Tails

Now we can return to our original question,

Example 8 What is the probability of flipping ten heads in thirty flips, with a fair coin?

Show solution

We broke it down into two parts:

$\begin{eqnarray*} P(h=10,N=30) = P\left(\text{\text{one sequence of 10 heads and 20 tails}}\right) \times \left(\text{number of rearrangements of a length-30 sequence with 10 ``H'' and 20 ``T''}\right) \end{eqnarray*}$

What is the probability of one particular sequence being considered?
$\begin{eqnarray*} P\left(\text{\text{one sequence of 10 heads and 20 tails}}\right)&=&\left(\frac{1}{2}\right)^{10}\times \left(\frac{1}{2}\right)^{20} \\ &=&\left(\frac{1}{2}\right)^{30} \\ &=&0.00000000093 \text{ (one in a billion!)} \end{eqnarray*}$
How many ways can this type of sequence appear in the process described in the question?
Because we have a length-thirty sequence of “H” and “T” with 10 duplicate “H” symbols and 20 duplicate “T,” we have the following number of ways that this could occur (i.e. the number of rearrangements of these sequences):
$\begin{eqnarray*} \left(\text{number of rearrangements of a length-30 sequence with 10 ``H'' and 20 ``T''}\right)&=&\frac{30!}{10!20!} \\ &=&30045015 \end{eqnarray*}$

So the probability of flipping 10 heads in 30 flips is

$\begin{eqnarray*} P(h=10,N=30)&=&\frac{30!}{10!20!}\left(\frac{1}{2}\right)^{30}\\ &=&30045015\times 0.00000000093 \\ &=&0.028 \end{eqnarray*}$

which is extremely unlikely (Table 1.1).

In general we have

Probability of flipping $h$ heads and $t$ tails Given the probability of flipping a single heads as 1/2, and the total number of flips is $N=h+t$ , we have the following equivalent forms:

$\begin{eqnarray} P(h,t)&=&\frac{(h+t)!}{h!t!}\times \left(\frac{1}{2}\right)^{h}\times \left(\frac{1}{2}\right)^{t}\\\label{eq:prob_heads} \nonumber P(h,N)&=&\frac{N!}{h!(N-h)!}\times \left(\frac{1}{2}\right)^{h}\times \left(\frac{1}{2}\right)^{N-h} \\ \nonumber P(h,N)&=&\binom{N}{h}\times \left(\frac{1}{2}\right)^{h}\times \left(\frac{1}{2}\right)^{N-h} \end{eqnarray}$

where we have introduced the notation that is sometimes used, called choose, read as “N choose h,”

$\begin{eqnarray*} \binom{N}{h}&\equiv&\frac{N!}{h!(N-h)!} \end{eqnarray*}$

Note

Probability of flipping $h$ heads and $t$ tails

Given the probability of flipping a single heads as 1/2, and the total number of flips is $N=h+t$ , we have the following probability for $h$ heads and $t$ tails:

$\begin{eqnarray*} P(h,t)=\frac{(h+t)!}{h!t!}\times \left(\frac{1}{2}\right)^{h}\times \left(\frac{1}{2}\right)^{t} \end{eqnarray*}$

Shown in Figure 3.1 is the probability of flipping $h$ heads in 30 flips, for each value of $h$ from $h=0$ (no heads or, in other words, 30 tails) up to $h=30$ (all 30 heads). Clearly the most likely value is 15, but all of the numbers from 12 up to 18 have significant probability.

Dotted curve of the probability of getting h heads in 30 flips of a fair coin. The distribution is bell-shaped, peaks at 15 heads with probability about 0.145, and is negligible below 5 or above 25 heads. — **Figure 3.1.** Probability of getting $h$ heads in 30 flips. Clearly the most likely value is 15, but all of the numbers from 12 up to 18 have significant probability.

Example 9 What is the probability of getting 17 or more heads in 30 flips?

Show solution

Because these are independent events, we can simply sum up the terms for $P(h=17,N=30)$ , $P(h=18,N=30)$ , etc... yielding the following, either through direct calculation, or by reading the Figure 3.1.

$\begin{eqnarray*} P(h\ge 17,N=30)&=&\underbrace{0.11}_{h=17} + \underbrace{0.08}_{h=18} + \underbrace{0.05}_{h=19} + \underbrace{0.028}_{h=20} + \underbrace{0.013}_{h=21} + \underbrace{0.005}_{h=22} + \underbrace{0.002}_{h=23} + \underbrace{\text{(tiny numbers)}}_{h=23, 24, 25, 26, 27, 28, 29, 30} \\ =0.29 \end{eqnarray*}$

which is quite likely!

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.

Choices	Remaining Symbols
A □ □ □	B C D
B □ □ □	A C D
C □ □ □	A B D
D □ □ □	A B C