7.5 Summary

It is useful to see all of these results stemming from the same Bayes' Recipe, applied to different models of the data and (possibly) different prior probabilities. As we have stated, many of the simple cases have been worked out by the mathematicians, so we don't need to do the work of deriving them. It will be our task to understand their properties, to be able to apply them to real problems, and to understand their consequences. One of the immediate observations that we make is the prevalence of the Normal distribution, justifying our detailed exploration of it in this chapter.

1. Proportions
   • Parameter of Interest: $\theta$, the chances of a single event
   • Applications: coin flips, voting percentages, success in sports, performance on tests
   • Form of the data: $h$ successes in $N$ total events
   • Model of the data:

     $$\begin{array}{c}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}=\{\begin{array}{cc}\hfill \mathrm{s}\mathrm{u}\mathrm{c}\mathrm{c}\mathrm{e}\mathrm{s}\mathrm{s}\hfill & \text{, with probability }\theta \hfill \\ \hfill \mathrm{f}\mathrm{a}\mathrm{i}\mathrm{l}\mathrm{u}\mathrm{r}\mathrm{e}\hfill & \text{, otherwise (i.e. with probability }1-\theta )\hfill \end{array}\end{array}$$

   • Posterior Probability:

     $$\begin{array}{c}\underset{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{B}\mathrm{e}\mathrm{t}\mathrm{a}(\theta |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})}}\sim \stackrel{\mathrm{l}\mathrm{i}\mathrm{k}\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{d}}{\overbrace{\mathrm{B}\mathrm{i}\mathrm{n}\mathrm{o}\mathrm{m}\mathrm{i}\mathrm{a}\mathrm{l}(\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}|\theta )}}\times \underset{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}(\theta )}}\end{array}$$

2. Magnitude with Known Deviation
   • Parameter of Interest: $\mu$, the true magnitude of a quantity, given the deviation, labeled by $\sigma$, from the central value
   • Applications: percentages with large samples, scientific measurements such as weight and size of objects, time scales of events
   • Form of the data: $N$ total data points, labeled $x_1,x_2,\cdots ,x_N$, and given known $\sigma$
   • Model of the data:

     $$\begin{array}{c}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}=\mu +\text{uncertainty with probability Normal(}\mu =0\text{,known }\sigma \text{)}\end{array}$$

   • Posterior Probability:

     $$\begin{array}{c}\underset{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}({\mu }_2|\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a},\sigma )}}\sim \stackrel{\mathrm{l}\mathrm{i}\mathrm{k}\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{d}}{\overbrace{\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}(\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}|\mu ,\sigma )}}\times \underset{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}(\mu )}}\end{array}$$

3. Magnitude with Unknown Deviation
   • Parameter of Interest: $\mu$, the true magnitude of a quantity, and the unknown deviation, labeled by $\sigma$, from the central value
   • Applications: scientific measurements with small samples (less than around 30), such as weight and size of objects, time scales of a small number of events
   • Form of the data: $N$ total data points, labeled $x_1,x_2,\cdots ,x_N$
   • Model of the data:

     $$\begin{array}{c}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}=\mu +\text{uncertainty with probability Normal(}\mu =0,\sigma \text{)}\end{array}$$

   • Posterior Probability:

     $$\begin{array}{c}\underset{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{P(\mu ,\sigma |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})}}\sim \stackrel{\mathrm{l}\mathrm{i}\mathrm{k}\mathrm{e}\mathrm{l}\mathrm{i}\mathrm{h}\mathrm{o}\mathrm{o}\mathrm{d}}{\overbrace{\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}(\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}|\mu ,\sigma )}}\times \underset{\mathrm{p}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}(\mu )·\mathrm{U}\mathrm{n}\mathrm{i}\mathrm{f}\mathrm{o}\mathrm{r}\mathrm{m}(\log \sigma )}}\end{array}$$

     $$\begin{array}{c}\underset{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{\mathrm{S}\mathrm{t}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}-\mathrm{T}(\mu |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})}}\sim {[P(\mu ,\sigma |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})]}_{\text{marginalized over }\sigma }\end{array}$$

     $$\begin{array}{c}\underset{\mathrm{p}\mathrm{o}\mathrm{s}\mathrm{t}\mathrm{e}\mathrm{r}\mathrm{i}\mathrm{o}\mathrm{r}\mathrm{p}\mathrm{r}\mathrm{o}\mathrm{b}\mathrm{a}\mathrm{b}\mathrm{i}\mathrm{l}\mathrm{i}\mathrm{t}\mathrm{y}}{\underbrace{F(\sigma |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})}}\sim {[P(\mu ,\sigma |\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a})]}_{\text{marginalized over }\mu }\end{array}$$

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.