Login
📚 Statistical Inference for Everyone
Chapters ▾
1 Introduction to Probability2 Applications of Probability3 Random Sequences and Visualization4 Introduction to Model Comparison5 Applications of Model Comparison6 Introduction to Parameter Estimation7 Priors, Likelihoods, and Posteriors8 Common Statistical Significance Tests9 Applications of Parameter Estimation and Inference10 Multi-parameter Models11 Introduction to MCMC12 Concluding Thoughts13 Appendix A: Computational Analysis14 Appendix B: Notation and Standards15 Appendix C: Common Distributions and Their Properties16 Appendix D: Tables

10.3 Multiple regression

In order to control for a variable that may be affecting our result, we simply expand our linear model, including slopes (also called coefficients) for each of the different variables. Once we do this, visualization becomes challenging because we move into three or more dimensions. Instead of m for the slope, the multiple slopes are typically labeled with the greek letter β and numbered, such as β1,β2, etc... The intercept is then labeled β0. The model structure, however, is the same, and can be written

y = β 0 + β 1 x 1 + β 2 x 2 ⋯

where the different x1, x2, etc... denote different variables. For the example of the SAT scores, we might have

t o t a l = β 0 + β 1 · e x p e n d i t u r e + β 2 · p e r c e n t _ t a k i n g

where percent_taking is a variable representing the percent of students taking the exam. Including this variable gives the posterior distributions shown in Figure 10.8. Notice that the effect of expenditure is both statistically significant and positive. We can interpret the values in the following way.

Have we controlled for all of the effects? Perhaps not! This is where the ingenuity and expertise of the person analyzing the problem comes into play.

Three stacked posterior curves from the two-variable SAT model: the expenditure coefficient centered at 12.29 (entirely positive, 1%-99% from 2.11 to 22.46), the percent-taking coefficient centered at -2.85 (entirely negative, -3.37 to -2.33), and the intercept centered at 993.83 (941.25 to 1046.41).
Figure 10.8. The posterior distributions for coefficients on the expenditure term, the percent taking term, and the intercept.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.