1.4 Conditional Probability

It is important to understand that probability reflects our state of knowledge about the system. As our knowledge changes, so do our probability assignments. As we gain more information, we change our probability assignments. Two people observing the same system, but with different information about the system, will give different probability assignments. All we need to make sure probability theory matches our common sense is for two people with the same state of knowledge, or the same information, to yield identical probability assignments.

Because our information about a system is so important in assigning probabilities, we introduce a way of writing it mathematically that we will use for the rest of the book. It will be good for the reader to get used to reading the mathematical short-hand in English in order to gain an understanding for what it means.

Probability Notation

In math, we choose to abbreviate long sentences in English, in order to use the economy of symbols. In this book we choose a middle-ground between mathematical succinctness and the ease of understanding English. We start with the simple card game (Equation 1.1)

We then define a new symbol, $|$ , which should be read as “given.” When there is information given we call this probability conditional on that information. When we write the following:

$P open parenthesis red on first draw vertical bar simple card game close parenthesis, (1.3)$

$P open parenthesis R sub 1 vertical bar simple card game close parenthesis, (1.4)$

this is short for

“The probability of drawing a red on the first draw, given that we have a standard initially shuffled deck and we follow the procedure where we draw one card, note what color it is and set it aside and continue drawing, noting, and setting aside until there are no more cards.”

One can easily see that the mathematical notation is far more efficient. It is important to be able to read the notation, because it describes what we know and what we want to know.

Conditional Probability When information is given, and expressed on the right-hand side of the $|$ sign, we say that the probability is conditional. $P (I'm going to get wet today | raining outside)$ is an assessment of how likely it is that I will get wet given, or conditional on, the fact that it is raining outside. Clearly this number will be different if it was conditional on the fact that it is sunny outside - different states of knowledge yield different probability assignments.

When we put a comma (“,”) on the right side then we read this as “and we know that.” For example, when we write the following:

Note

Causation. Imagine we have a 2-card game: a small deck with one red card and one black card, and I draw a red card first. Clearly this makes the probability of drawing red as the second card equal to zero - it can't happen. We're tempted to interpret

$P open parenthesis R sub 2 vertical bar R sub 1, 2-card game close parenthesis equals 0$

to mean that because we drew a red on the first draw, this causes the impossibility of drawing the red on the second - there is only 1 red card after all, and drawing it seems to cause the impossibility of drawing red in the future. However, consider the following:

$P open parenthesis R sub 1 vertical bar R sub 2, 2-card game close parenthesis equals 0$

which is, if we knew that the second card we drew was red, then it makes it impossible to have drawn a red card as the first card. This is just as true as the previous case, however, you can't interpret this as causation - the second draw didn't cause the first draw.

Instead, probability statements are statements of logic, not causation. One can use probabilities to describe causation (i.e. $P (rain | clouds)$ ), but the statements of probability have no time component - later draws from the deck of cards act exactly the same as earlier ones.

$P open parenthesis red on second draw vertical bar simple card game,red on first draw close parenthesis, (1.5)$

$P open parenthesis R sub 2 vertical bar simple card game, R sub 1 close parenthesis, (1.6)$

this is short for

“The probability of drawing a red on the second draw, given that we have a standard initially shuffled deck and we follow the procedure where we draw one card, note what color it is and set it aside and continue drawing, noting, and setting aside until there are no more cards and we know that we drew a red on the first draw.”

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.