2.9 Some Philosophical Applications

Doctors' Claims - English Language and Probability

In Section 1.5 we introduced work by Tversky and Kahneman documenting supposed failures in proper reasoning. In the example survey of medical internists, the internists were asked

Which is more likely: the victim of an embolism (clot in the lung) will experience partial paralysis or that the victim will experience both partial paralysis and shortness of breath?

and 91 percent of the doctors chose that the clot was less likely to cause the rare paralysis rather than to cause the combination of the rare paralysis and the common shortness of breath.

This may not be a failure of reasoning, but a (correct!) failure of the doctors to translate the English language literally into logical language. It is likely that when doctors are asked: “Which is more likely: that the victim of an embolism will experience partial paralysis or that the victim will experience both partial paralysis and shortness of breath?” they interpret it as:

someone is claiming that the patient has an embolism
the patient is claiming, or someone has measured, that she has partial paralysis
the patient is claiming, or someone has measured, that she has shortness of breath

The doctors are separating the analysis of the claim of the clot, which is given information, from the other claims. Another way of looking at it is to include the knowledge of the method of reporting. Someone who is reporting information about an ailment will tend to report all of the information accessible to them. By reporting only the paralysis, there are two possibilities concerning the person measuring the symptoms of the patient:

they had the means to measure shortness breath in the patient, but there was none
they did not have the means to measure shortness of breath

In the first case, the doctor's probability assessment is absolutely correct: both symptoms together are more likely than just one. In the second case, the doctors are also correct: one of the sets of diagnostic results (i.e. just paralysis) is less dependable than the other set (i.e. both symptoms), thus the second one is more likely to indicate a clot or is consistent with the known clot.

It isn't that the doctors are reasoning incorrectly. They are including more information, and doing a more sophisticated inference than the strict, formal, minimalistic interpretation of the statements would lead one to do. This analysis works well for other examples stated in the book A Drunkard's Walk by Mlodinow, like “Is it more probable that the president will increase federal aid to education or that he or she will increase federal aid to education with funding freed by cutting other aid to states?”

All of this underscores the need to be careful translating statements of probability into plain English and vice versa.

Diverging Opinions

Is it possible to have people informed by the same information, and reasoning properly, to have diverging opinions? It might seem intuitive that people given the same information, reasoning properly, would tend to come to agreement, however this is not always the case. What is interesting is that it turns on the prior probabilities for claims. This example comes from Jaynes, 2003¹. We have the following piece of information:

$\begin{matrix} D : = {``Mr N . has gone on TV with a sensational claim that a commonly used drug is unsafe'' \end{matrix}$

and we have observers $A$ , $B$ , and $C$ with different prior assignments to the reliability of Mr $N$ and of the safety of the drug. These prior assignments may have been the result of previous inference by these observers, in a different context, or possibly due to expert knowledge. Observers $A$ and $C$ believe, before the announcement, that the drug is reasonably safe. Observer $B$ does not. We have the probability assignments then:

$\begin{matrix} P_{A} (S a f e) = 0.9 \\ P_{B} (S a f e) = 0.1 \\ P_{C} (S a f e) = 0.9 \end{matrix}$

They all agree that if the drug is not safe, then Mr $N$ would announce it, so we have

$\begin{matrix} P_{A} (D | not Safe) = 1 \\ P_{B} (D | not Safe) = 1 \\ P_{C} (D | not Safe) = 1 \end{matrix}$

Finally, we have the perceptions from the observers about the reliability of Mr $N$ if the drug is actually safe. In this case, observer $A$ is trusting of Mr $N$ , observer $C$ is strongly distrustful, and observer $B$ is mildly distrustful. By “distrustful” we are referring to the probabilities that Mr $N$ would make the announcement that the drug is unsafe even if the drug were actually safe. So we have

$\begin{matrix} P_{A} (D | Safe) = 0.01 \\ P_{B} (D | Safe) = 0.3 \\ P_{C} (D | Safe) = 0.99 \end{matrix}$

We want to know how each observer then determines whether the drug is safe, given the announcement, or $P (Safe | D)$ for each observer.

Applying Bayes' Rule we have

$\begin{matrix} P_{A} (Safe | D) = \frac{P_{A} (D | Safe) P_{A} (Safe)}{P_{A} (D | Safe) P_{A} (Safe) + P_{A} (D | not Safe) P_{A} (not Safe)} \\ = \frac{0.01 \cdot 0.9}{0.01 \cdot 0.9 + 1 \cdot 0.1} = 0.083 \end{matrix}$

Following the same calculation for the others, we get the observers updating their probability assignments after the announcement, $D$ , as

$\begin{matrix} P_{A} (S a f e) = 0.9 \to P_{A} (Safe | D) = 0.083 \\ P_{B} (S a f e) = 0.1 \to P_{B} (Safe | D) = 0.032 \\ P_{C} (S a f e) = 0.9 \to P_{C} (Safe | D) = 0.899 \end{matrix}$

Observer $A$ changed their mind, Observer $B$ had their assessment confirmed a bit, and Observer $C$ barely budged.

Although you'd think that hearing the announcement of the unsafe nature of the drug would have moved all of the probabilities by the same amount, but the information isn't that the drug is unsafe, but the someone is claiming that the drug is unsafe. Thus, ones prior information about both the drug and who is making the claim comes into play.

A problem of independence

As said in the beginning of Chapter 1 (Introduction to Probability), in 1968 a jury found defendant Malcolm Ricardo Collins and his wife defendant Janet Louise Collins guilty of second degree robbery. The prosecutor focussed on the the distinctive features of the dependence, and assigned a probability to each as follows²:

Partly yellow automobile 1/10
Man with mustache 1/4
Girl with ponytail 1/10
Girl with blond hair 1/3
Negro man with beard 1/10
Interracial couple in car 1/1000

He then followed with the calculation applying the product rule for independent events (Section 1.5 on page 43), to find the probability that all these things could have been observed:

$\begin{matrix} \frac{1}{10} \times \frac{1}{4} \times \frac{1}{10} \times \frac{1}{3} \times \frac{1}{10} \times \frac{1}{1000} = \frac{1}{12, 000, 000} \end{matrix}$

The initial conviction was overturned for two primary reasons, one legal and one mathematical. The legal argument was that the prosecution had not established that these initial probabilities were supported by the evidence. However, the really devastating part of the argument was mathematical. As you may recall, the product rule used in this way assumes the independence of the terms (Section 1.5 on pageSection 43).

Example 1 Beard and Mustache - An Examination of Independence

For an example, the proper product rule for two of the terms above would look like:

$\begin{matrix} P (Man with beard and Man with mustache) = \\ P (Man with mustache | Man with beard) P (Man with beard) \end{matrix}$

What the prosecutor was assuming is that these two items were independent, from which it would follow that

$\begin{matrix} P (Man with beard and Man with mustache) = \\ P (Man with mustache) P (Man with beard) = \frac{1}{40} = 0.025 \end{matrix}$

However, with a very brief thought, we notice that this is equivalent to saying

Knowing the man has a beard tells us nothing about the probability of him having a mustache!

Clearly, it is not nearly as common to have a beard with no mustache than with one, so knowing that the man had a beard would nearly certainly imply that he had a mustache or,

$\begin{matrix} P (Man with beard and Man with mustache) = \\ \underset{\sim 1}{\underset{⏟}{P (Man with mustache | Man with beard)}} P (Man with beard) \sim \frac{1}{10} \end{matrix}$

and the probability calculated, just from these two terms, is much higher than the prosecutor was communicating.

Similar sorts of absurdities occur with other terms, like “blond hair” and “pony tail”, as well as others. Finally, even if it was the case that this is a somewhat rare combination, given the number of people in Los Angeles, one might be able to calculate the probability that there is at least one more couple satisfying these characteristics. Just like the lottery problem (Section 2.6 on page 62), it becomes likely that there are more couples in the area like this, and thus the ruling was overturned.

Another problem with independence

Another problem brought up in the opening of Chapter 1 (Introduction to Probability) is the case of Sally Clark. Sally Clark was convicted in 1999 of the murder of her two young sons³. In the case, the statistical argument was

Professor Meadow was asked if a figure of 1 in 8,543 reflected the risk of there being a single SIDS within such a family. He agreed that it was. A table from the CESDI report was placed before the jury. He was then asked if the report calculated the risk of two infants dying of SIDS in that family by chance. His reply was: ‘‘Yes, you have to multiply 1 in 8,543 times 1 in 8,543 and I think it gives that in the penultimate paragraph. It points out that it's approximately a chance of 1 in 73 million.”

What he was doing was equating the following in the product rule (Section 1.5 on page 1.5):

$\begin{matrix} P (second child dying of SIDS | first child dying of SIDS) = P (second child dying of SIDS) \end{matrix}$

which is equivalent to saying

Knowing that the child dies of a [not well understood] disease tells us nothing about the probability of the second child dying of the same [not well understood] disease.

Clearly this is ridiculous, because if there is a common source to the disease, the one death certainly increases the probability of the second. Such a common source could be something shared in the environmental or perhaps a genetic disposition in the family for the disease.

Prosecutor's Fallacy

Both of the cases above are examples of what is called the prosecutor's fallacy. It occurs when someone assumes that the prior probability of an event is equal to the probability that the defendant is innocent. A simple example is that “if a perpetrator is known to have the same blood type as a defendant and 10% of the population share that blood type; then to argue on that basis alone that the probability of the defendant being guilty is 90% makes the prosecutors's fallacy, in a very simple form.”

Essentially the prosecutor is ignoring the number of people who match the rare event. Also, although double-deaths by SIDS are rare, they are much more common than double-murders! One really has to look at

$\begin{matrix} P (innocence | evidence) \end{matrix}$

which is not the same as

$\begin{matrix} P (evidence) \end{matrix}$

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.