1.7 Lessons from Bayes' Rule - A First Look

Bayes' Rule is the gold standard for all statistical inference. It is a mathematical theorem, proven from fundamental principles. It structures all inference in a systematic fashion. However, it can be used without doing any calculations, as a guide to qualitative inference. Some of the lessons which are consequences of Bayes' Rule are listed here, and will be noted throughout this text in various examples.

• Confidence in a claim should scale with the evidence for that claim
• Ockham's razor, which is the philosophical idea that simpler theories are preferred, is a consequence of Bayes' Rule when comparing models of differing complexity.
• Simpler means fewer adjustable parameters
• Simpler also means that the predictions are both specific and not overly plastic. For example, a hypothesis which is consistent with the observed data, and also be consistent if the data were the opposite would be overly plastic.
• Your inference is only as good as the hypotheses (i.e. models) that you consider.
• Extraordinary claims require extraordinary evidence.¹
• It is better to explicitly display your assumptions rather than implicitly hold them.
• It is a good thing to update your beliefs when you receive new information.
• Not all uncertainties are the same.

There is not a universal agreement for the translation of numerical probability values to qualitative terms in English (i.e. highly unlikely, somewhat unlikely, etc...). One rough guide is shown in Table 1.1. I will be following this convention throughout the book, but realize that the specific probability distinctions are a bit arbitrary.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.