2.2 Cancer and Probability

This is perhaps the most important probability question to learn, so we will spend some time covering it here and then cover it again, in a slightly different way, in Section 5.2 on page 109. Imagine we have a population of 10000 people who have been tested for cancer, and we get the following hypothetical data:

Number of Individuals	Negative Test	Positive Test	Total
Doesn't Have Cancer	9200	700	9900
Has Cancer	20	80	100
	9220	780	10000

We may be interested in a number of related probabilities.

Example 1 What is the probability of both having cancer and getting a positive test for it?

Show solution

We can determine this by simply dividing the person counts from the table

$\begin{eqnarray*} P\left(\text{cancer \textbf{and} positive test}\right) &=& \frac{\text{ of people with both cancer and positive test}}{\text{total  of people}} \\ &=&\frac{80}{10000} = 0.008 \end{eqnarray*}$

Doing this process for every part of the table yields a posterior probability table, giving the probability for every combination of variables (i.e. with cancer and positive test, without cancer and positive test, etc...)

Posterior Probability	Negative Test	Positive Test	Total
Doesn't Have Cancer	0.92	0.07	0.99
Has Cancer	0.002	0.008	0.01
	0.922	0.078	1.0

Example 2 What is the probability of both not having cancer and getting a positive test for it?

Show solution

Reading off of the table, we have

$\begin{eqnarray*} P\left(\text{no cancer \textbf{and} positive test}\right) &=& 0.07 \end{eqnarray*}$

This question, it turns out, is a not very interesting question. The type of question that actually arises in life is the following,

Example 3 What is the probability of having cancer given a positive test for it?

Show solution

Here we can perform the calculation in a couple of different ways, to give the (unintuitive) result.

Counting the individuals.
$\begin{eqnarray*} P\left(\text{cancer} \,|\, \text{positive test}\right) &=& \frac{\text{ of people with both cancer and positive test}}{\text{ of people with a positive test}} \\ &=&\frac{80}{780} = 0.103 \end{eqnarray*}$
Although those with cancer nearly always test positive, out of the pool of all people who test positive - including a large number of false-positives - those actually having cancer are a small minority. It is because there are many more people without cancer, so even if a small fraction of those mistakenly test positive it will outweigh the small fraction of those people with the disease. This is why we insist on second opinions and why the rarity of a disease often matters even more than the accuracy of the test.
Applying Product Rule
Using the Product Rule (Section 1.5 on page 42), we have

$\begin{eqnarray*} P\left(\text{positive test}\right) &=& P\left(\text{no cancer \textbf{and} positive test}\right) + P\left(\text{cancer \textbf{and} positive test}\right) \\ &=&= 0.07 + 0.008 = 0.078\\ P\left(\text{cancer} \,|\, \text{positive test}\right) &=& \frac{P\left(\text{cancer \textbf{and} positive test}\right)}{P\left(\text{positive test}\right)} \\ &=& \frac{0.008}{0.078} = 0.103 \end{eqnarray*}$

where we have used the sum of the Positive Test column for $P\left(\text{positive test}\right)$ . This is simply a shortcut to the marginalization process (Section 1.5 on page 46) - determine the probability of an event by adding up all of the possible conditional situations.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.