Chapter 7: Priors, Likelihoods, and Posteriors

In Chapter 6 (Introduction to Parameter Estimation on page 121) we estimated the chance, $\theta$, that a bent coin would come up heads by combining a uniform prior for $\theta$ (i.e. all possible values are a-priori equally likely) and a binomial likelihood (i.e. given $\theta$, what is the probability of the data). This resulted in a Beta distribution for the posterior probability for $\theta$.

Notice what the procedure of Bayes' Recipe is and how the Bayesian inference works here.

1. Specify the prior probabilities for the models being considered

   We want to estimate a quantity (which we label as $\theta$), but begin with absolutely no knowledge of its value - we have a uniform prior probability.

2. Write the top of Bayes' Rule for all models being considered

   We construct a model for how different possible values of $\theta$ influence the outcome - a model we call the likelihood. In the case of the bent coin, the likelihood model is a binomial model, and describes the probability of flipping heads or tails given how bent the coin is (i.e. given $\theta$).