6.9 Marginalization

In Section 1.5 we introduced the concept of marginalization, and in Section 2.2 we performed a discrete example of this. In that section it was seen as simply a consequence of the sum and product rules. It was a way of taking a probability that depended on several factors, and eliminating all but the single factor we're interested in. If we have a continuous distribution, this process involves calculus and we will not cover it in detail, but it is the same process. In the case of the distribution above, we have a distribution over a single variable, like $\mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}(\theta |h,t)$. Imagine that we have a distribution that depends on two parameters,

$$\begin{array}{c}\mathrm{M}\mathrm{y}\mathrm{D}\mathrm{i}\mathrm{s}\mathrm{t}(\theta ,\xi )\end{array}$$

which specifies the probability of an event given each combination of the parameters, $\theta$ and $\xi$. We'd have to do a three-dimensional plot to visualize this. Many times, however, we want just the probability of one of the single parameters. In those cases we will write

$$\begin{array}{c}P(\theta )\sim {[\mathrm{M}\mathrm{y}\mathrm{D}\mathrm{i}\mathrm{s}\mathrm{t}(\theta ,\xi )]}_{\text{marginalize over }\xi }\end{array}$$

where we are “summing” over all the values of the other parameters, leaving the details to the mathematicians, and simply using the result.

Likewise we can marginalize the parameter $\theta$ to get the distribution of the other variable.

$$\begin{array}{c}P(\xi )\sim {[\mathrm{M}\mathrm{y}\mathrm{D}\mathrm{i}\mathrm{s}\mathrm{t}(\theta ,\xi )]}_{\text{marginalize over }\theta }\end{array}$$

This becomes important in Chapter 7 and Chapter 9.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.