Login
📚 Statistical Inference for Everyone
Chapters ▾
1 Introduction to Probability2 Applications of Probability3 Random Sequences and Visualization4 Introduction to Model Comparison5 Applications of Model Comparison6 Introduction to Parameter Estimation7 Priors, Likelihoods, and Posteriors8 Common Statistical Significance Tests9 Applications of Parameter Estimation and Inference10 Multi-parameter Models11 Introduction to MCMC12 Concluding Thoughts13 Appendix A: Computational Analysis14 Appendix B: Notation and Standards15 Appendix C: Common Distributions and Their Properties16 Appendix D: Tables

8.1 Overview

The basic idea of common statistical tests in the approach we have taken has been the following:

  1. Observe some data
  2. Construct a model of the data, with a parameter that needs to be estimated, such as the “true” single value (μ, in Section 7.3), or the proportion of the event (θ, in Section 7.4).
  3. Calculate the final, posterior probability of that parameter
  4. “Test” to see if there is a significant (usually 95%) probability that the parameter is not zero.
  5. If the test passes, then one can be reasonably confident that the parameter is non-zero - that the effect is real. If the test fails, then under the model, the possibility of a zero-effect cannot be reasonably excluded.

These tests are a subset of the parameter estimation techniques covered in both Chapter 6 (Introduction to Parameter Estimation on page 121) and Chapter 7 (Priors, Likelihoods, and Posteriors on page 139), in the special case where we are interested in determining if there is an effect at all. For example, we might be interested to see if a medical treatment works, so we compare the before- and after-treatment values to see if the difference is non-zero.

The tests that one typically employs in simple cases go by various names, depending on the model. This chapter summarizes several of the common ones, and applies them to some typical cases.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.