#set document(title: "12.5 Hypothesis Test for Linear Regression", author: "Rachel Webb") #set page(width: 8.5in, height: auto, margin: 1in) #import "@preview/cetz:0.5.2" #set text(font: ("STIX Two Text", "Libertinus Serif", "New Computer Modern"), size: 10.5pt, lang: "en") #show math.equation: set text(font: ("STIX Two Math", "New Computer Modern Math")) #set par(justify: true, leading: 0.62em, spacing: 0.9em) #set enum(spacing: 1.1em) // room between list items so tall inline fractions don't collide #set list(spacing: 1.1em) #set table(stroke: 0.5pt + rgb("#c7ccd3")) #let BLUE = rgb("#183B6F") // brand navy — section bars + example/solution labels (white on navy 11.09:1) #let ORANGE = rgb("#A94509") // brand primary-700 — AA-safe deep orange for TEXT (5.93:1 on white; raw brand #F37021 is 2.94:1 and must never carry text) #let RED = rgb("#DC2626") // brand error-600 #let GREEN = rgb("#059669") // brand success-600 (decoration only; small green text uses green-text #007942) #show heading.where(level: 1): it => block(width: 100%, above: 0pt, below: 16pt, fill: gradient.linear(BLUE, rgb("#2C5AA0")), inset: (x: 14pt, y: 12pt), radius: 3pt, text(fill: white, weight: "bold", size: 19pt, it.body)) #show heading.where(level: 2): it => block(width: 100%, above: 18pt, below: 10pt, fill: BLUE, inset: (x: 10pt, y: 6pt), radius: 2pt, text(fill: white, weight: "bold", size: 12pt, it.body)) #show heading.where(level: 3): it => text(fill: ORANGE, weight: "bold", size: 12.5pt, it.body) #show heading.where(level: 4): it => text(fill: BLUE, weight: "bold", size: 10.5pt, it.body) #let examplebox(label, title, body) = block(width: 100%, breakable: true, fill: rgb("#EFF1F5"), stroke: 0.5pt + rgb("#CFDDF0"), radius: 4pt, inset: 10pt, above: 12pt, below: 12pt)[ #block(below: 6pt)[#box(fill: BLUE, inset: (x: 6pt, y: 2pt), radius: 2pt, text(fill: white, weight: "bold", size: 8.5pt, label)) #h(0.4em) #strong[#title]] #body] // rail = decorative left rule (raw brand token); labelcolor = AA-safe label text shade #let notebox(label, rail, labelcolor, tint, body) = block(width: 100%, breakable: true, fill: tint, stroke: (left: 3pt + rail), inset: (left: 10pt, rest: 8pt), radius: (right: 4pt), above: 11pt, below: 11pt)[ #text(fill: labelcolor, weight: "bold", size: 7.5pt, tracking: 0.5pt)[#upper(label)] #linebreak() #body] #let solutionbox(body) = block(above: 4pt, below: 8pt)[ #text(fill: BLUE, weight: "bold", size: 8.5pt)[Solution] #linebreak() #body] #let figph(msg) = block(width: 100%, height: 60pt, fill: rgb("#f6f7f9"), stroke: (paint: rgb("#c7ccd3"), dash: "dashed"), radius: 4pt, inset: 10pt)[ #align(center + horizon, text(fill: rgb("#889"), style: "italic", size: 9pt, msg))] // Standardize inlined figure sizes: measure the natural CeTZ canvas, then scale to a // consistent envelope (aspect-aware; see build_typst.py FIG_* constants). Unlike the // print preamble, dimensions are FLOORED: in an editor a user can trim a figure to a // degenerate 1-D shape (a bare line), and w/h or tw/w would then divide by zero. #let _STD_W = 3.5 #let _WIDE_W = 5.6 #let _MAX_H = 3.4 #let _ASPECT_WIDE = 2.2 #let _UPSCALE_MAX = 1.15 #let stdfig(body) = context { let m = measure(body) let w = calc.max(m.width / 1in, 0.01) let h = calc.max(m.height / 1in, 0.01) let tw = if w / h > _ASPECT_WIDE { _WIDE_W } else { _STD_W } let s = calc.min(tw / w, _MAX_H / h, _UPSCALE_MAX) align(center, box(scale(x: s * 100%, y: s * 100%, reflow: true, body))) } #show figure: set block(breakable: false) #set figure(gap: 8pt) #show figure.caption: set text(size: 8.5pt, fill: rgb("#555")) == 12.5#h(0.6em)Hypothesis Test for Linear Regression To test to see if the slope is significant we will be doing a two-tailed test with hypotheses. The population least squares regression line would be #math.equation(block: false, alt: "y equals β sub 0 plus β sub 1 plus ε")[$y = β_(0) + β_(1) + ε$] where #math.equation(block: false, alt: "β sub 0")[$β_(0)$] (pronounced “beta-naught”) is the population #math.equation(block: false, alt: "y")[$y$]-intercept, #math.equation(block: false, alt: "β sub 1")[$β_(1)$] (pronounced “beta-one”) is the population slope and #math.equation(block: false, alt: "ε")[$ε$] is called the error term. If the slope were horizontal (equal to zero), the regression line would give the same #math.equation(block: false, alt: "y")[$y$]-value for every input of #math.equation(block: false, alt: "x")[$x$] and would be of no use. If there is a statistically significant linear relationship then the slope needs to be different from zero. We will only do the two-tailed test, but the same rules for hypothesis testing apply for a one-tailed test. We will only be using the two-tailed test for a population slope. The hypotheses are: #math.equation(block: true, alt: "H sub 0 : β sub 1 equals 0")[$H_(0) : β_(1) = 0$] #linebreak() #math.equation(block: true, alt: "H sub 1 : β sub 1 not equal to 0")[$H_(1) : β_(1) ≠ 0$] The null hypothesis of a two-tailed test states that there is not a linear relationship between #math.equation(block: false, alt: "x")[$x$] and #math.equation(block: false, alt: "y")[$y$]. The alternative hypothesis of a two-tailed test states that there is a significant linear relationship between #math.equation(block: false, alt: "x")[$x$] and #math.equation(block: false, alt: "y")[$y$]. Either a t-test or an F-test may be used to see if the slope is significantly different from zero. The population of the variable #math.equation(block: false, alt: "y")[$y$] must be normally distributed. === F-Test for Regression An F-test can be used instead of a t-test. Both tests will yield the same results, so it is a matter of preference and what technology is available. Figure 12-12 is a template for a regression ANOVA table, #figure(figph[Template for a regression table, containing equations for the sum of squares, degrees of freedom and mean square for regression and for error, as well as the F value of the data.], alt: "Template for a regression table, containing equations for the sum of squares, degrees of freedom and mean square for regression and for error, as well as the F value of the data.", caption: [Figure 12-12: Regression ANOVA table template.]) where #math.equation(block: false, alt: "n")[$n$] is the number of pairs in the sample and #math.equation(block: false, alt: "p")[$p$] is the number of predictor (independent) variables; for now this is just #math.equation(block: false, alt: "p equals 1")[$p = 1$]. Use the F-distribution with degrees of freedom for regression = #math.equation(block: false, alt: "d f sub R equals p")[$d f_(R) = p$], and degrees of freedom for error = #math.equation(block: false, alt: "d f sub E equals n minus p minus 1")[$d f_(E) = n − p − 1$]. This F-test is always a right-tailed test since ANOVA is testing the variation in the regression model is larger than the variation in the error. #examplebox("Example 1")[][ Use an F-test to see if there is a significant relationship between hours studied and grade on the exam. Use #math.equation(block: false, alt: "α")[$α$] = 0.05. #notebox("Note", rgb("#8a94a6"), rgb("#556666"), rgb("#f7f8fa"))[ #emph[The F-test's critical value and p-value] Both links open the F-distribution with df = 1 and 13 - the same curve as the book's online applet, no external site needed. The first gives the critical value 4.667 (left-tail area 0.95, the book's =F.INV.RT(0.05,1,13)); the second shades the right tail above the test statistic F = 27.78 to give the p-value 0.00015. - Critical value: F.INV.RT(0.05, 1, 13) = 4.667 - p-value: P(F \> 27.78) = 0.00015 ] Hours Studied for Exam 20 16 20 18 17 16 15 17 15 16 15 17 16 17 14 Grade on Exam 89 72 93 84 81 75 70 82 69 83 80 83 81 84 76 #solutionbox[ The hypotheses are: #math.equation(block: true, alt: "H sub 0 : β sub 1 equals 0")[$H_(0) : β_(1) = 0$] #linebreak() #math.equation(block: true, alt: "H sub 1 : β sub 1 not equal to 0")[$H_(1) : β_(1) ≠ 0$] Compute the sum of squares. #math.equation(block: false, alt: "S S sub x x equals 41.6")[$S S_(x x) = 41.6$], #math.equation(block: false, alt: "S S sub y y equals 631.7333")[$S S_(y y) = 631.7333$], #math.equation(block: false, alt: "S S sub x y equals 133.8")[$S S_(x y) = 133.8$], #math.equation(block: false, alt: "n equals 15")[$n = 15$] and #math.equation(block: false, alt: "p equals 1")[$p = 1$] #math.equation(block: true, alt: "S S R equals the fraction open parenthesis S S sub x y close parenthesis squared over S S sub x x equals the fraction open parenthesis 133.8 close parenthesis squared over 41.6 equals 430.3471154")[$S S R = frac(attach(( S S_(x y) ), t: 2), S S_(x x)) = frac(( 133.8 )^(2), 41.6) = 430.3471154$] #math.equation(block: true, alt: "S S E equals S S T minus S S R equals 631.7333 minus 430.3471154 equals 201.3862")[$S S E = S S T − S S R = 631.7333 − 430.3471154 = 201.3862$] #math.equation(block: true, alt: "S S T equals S S sub y y equals 631.7333")[$S S T = S S_(y y) = 631.7333$] Compute the degrees freedom. #math.equation(block: true, alt: "d f sub T equals n minus 1 equals 14 d f sub E equals n minus p minus 1 equals 15 minus 1 minus 1 equals 13")[$d f_(T) = n − 1 = 14 #h(1em) #h(1em) d f_(E) = n − p − 1 = 15 − 1 − 1 = 13$] Compute the mean squares. #math.equation(block: true, alt: "M S R equals the fraction S S R over p equals the fraction 430.3471154 over 1 equals 430.3471154 M S E equals the fraction S S E over n minus p minus 1 equals the fraction 201.3862 over 13 equals 15.4912")[$M S R = frac(S S R, p) = frac(430.3471154, 1) = 430.3471154 #h(1em) #h(1em) M S E = frac(S S E, n − p − 1) = frac(201.3862, 13) = 15.4912$] Compute the Test Statistic #math.equation(block: true, alt: "F equals the fraction M S R over M S E equals the fraction 430.3471154 over 15.4912 equals 27.7801")[$F = frac(M S R, M S E) = frac(430.3471154, 15.4912) = 27.7801$] Substitute the numbers into the ANOVA table: #figure(figph[Regression ANOVA table containing the values calculated above.], alt: "Regression ANOVA table containing the values calculated above.", caption: none) This is a right-tailed F-test with #math.equation(block: false, alt: "d f equals 1 , 13")[$d f = 1 , 13$] and #math.equation(block: false, alt: "α")[$α$] = 0.05, which gives a critical value of 4.667. In Excel we can find the critical value by using the function =F.INV.RT(0.05,1,13) = 4.667. #figure(figph[Graph of the F-distribution with right tail, starting at the critical value of 4.667, shaded in.], alt: "Graph of the F-distribution with right tail, starting at the critical value of 4.667, shaded in.", caption: none) Or use the online calculator at #link("https://homepage.divms.uiowa.edu/~mbognar/applets/f.html")[https://homepage.divms.uiowa.edu/~mbognar/applets/f.html] to visualize the critical value, as shown in Figure 12-13. It is hard to see the shaded tail in the following picture above the test statistic since the F-distribution is so close to the #math.equation(block: false, alt: "x")[$x$]-axis after 3, but it has the right-tail shaded from 4.667 and greater. #figure(figph[F-distribution generated by an online calculator with inputs of a df\_1 value of 1, a df\_2 value of 13, a critical value of 4.66719, and an alpha of 0.05.], alt: "F-distribution generated by an online calculator with inputs of a df_1 value of 1, a df_2 value of 13, a critical value of 4.66719, and an alpha of 0.05.", caption: [Figure 12-13: F-distribution graph generated by online calculator, with the input values displayed.]) The test statistic 27.78 is even further out in the tail than the critical value, so we would reject #math.equation(block: false, alt: "H sub 0")[$H_(0)$]. At the 5% level of significance, there is a statistically significant relationship between hours studied and grade on a student’s final exam. The p-value could also be used to make the decision. The p-value method would use the function =F.DIST.RT(27.78,1,13) = 0.00015 in Excel. The p-value is less than #math.equation(block: false, alt: "α")[$α$] = 0.05, which also verifies that we reject #math.equation(block: false, alt: "H sub 0")[$H_(0)$]. The following is the output from Excel and SPSS. Note the same ANOVA table information is shown but the columns are in a different order. #strong[Excel] #linebreak() #linebreak() #strong[SPSS] #linebreak() ] ] === T-Test for Regression If the regression equation has a slope of zero, then every #math.equation(block: false, alt: "x")[$x$] value will give the same #math.equation(block: false, alt: "y")[$y$] value and the regression equation would be useless for prediction. We should perform a t-test to see if the slope is significantly different from zero before using the regression equation for prediction. The numeric value of t will be the same as the t-test for a correlation. The two test statistic formulas are algebraically equal; however, the formulas are different and we use a different parameter in the hypotheses. The formula for the t-test statistic is #math.equation(block: false, alt: "t equals the fraction b sub 1 over the square root of open parenthesis the fraction M S E over S S sub x x close parenthesis")[$t = frac(b_(1), sqrt(( frac(M S E, S S_(x x)) )))$] Use the t-distribution with degrees of freedom equal to #math.equation(block: false, alt: "n minus p minus 1")[$n − p − 1$]. The t-test for slope has the same hypotheses as the F-test: #math.equation(block: true, alt: "H sub 0 : β sub 1 equals 0")[$H_(0) : β_(1) = 0$] #linebreak() #math.equation(block: true, alt: "H sub 1 : β sub 1 not equal to 0")[$H_(1) : β_(1) ≠ 0$] #examplebox("Example 2")[][ Use a t-test to see if there is a significant relationship between hours studied and grade on the exam, use #math.equation(block: false, alt: "α")[$α$] = 0.05. Hours Studied for Exam 20 16 20 18 17 16 15 17 15 16 15 17 16 17 14 Grade on Exam 89 72 93 84 81 75 70 82 69 83 80 83 81 84 76 #solutionbox[ The hypotheses are: #math.equation(block: true, alt: "H sub 0 : β sub 1 equals 0")[$H_(0) : β_(1) = 0$] #linebreak() #math.equation(block: true, alt: "H sub 1 : β sub 1 not equal to 0")[$H_(1) : β_(1) ≠ 0$] Find the critical value using #math.equation(block: false, alt: "d f sub E equals n minus p minus 1 equals 13")[$d f_(E) = n − p − 1 = 13$] for a two-tailed test #math.equation(block: false, alt: "α")[$α$] = 0.05 inverse t-distribution to get the critical values #math.equation(block: false, alt: "± 2.160")[$± 2.160$]. Draw the sampling distribution and label the critical values, as shown in Figure 12-14. #figure(figph[Graph of the t-distribution with both tails shaded in, starting at the critical values of positive and negative 2.160.], alt: "Graph of the t-distribution with both tails shaded in, starting at the critical values of positive and negative 2.160.", caption: [Figure 12-14: Graph of t-distribution with labeled critical values.]) The critical value is the same as we found using the t-test for correlation. Next, find the test statistic #math.equation(block: false, alt: "t equals the fraction b sub 1 over the square root of open parenthesis the fraction M S E over S S sub x x close parenthesis equals the fraction 3.216346 over the square root of open parenthesis the fraction 15.4912 over 41.6 close parenthesis")[$t = frac(b_(1), sqrt(( frac(M S E, S S_(x x)) ))) = frac(3.216346, sqrt(( frac(15.4912, 41.6) )))$] = 5.271\\). The test statistic value is the same value of the t-test for correlation even though they used different formulas. We look in the same place using technology as the correlation test. #figure(figph[Using a calculator to conduct a LinRegTTest and find the value for t.], alt: "Using a calculator to conduct a LinRegTTest and find the value for t.", caption: none) The test statistic is greater than the critical value of 2.160 and in the rejection region. The decision is to reject #math.equation(block: false, alt: "H sub 0")[$H_(0)$]. Summary: At the 5% significance level, there is enough evidence to support the claim that there is a significant linear relationship (correlation) between the number of hours studied for an exam and exam scores. The p-value method could also be used to find the same decision. The p-value = 0.00015, the same as the previous tests. We will use technology for the p-value method. In the SPSS output, they use Sig. for the p-value. #strong[Excel] #linebreak() #strong[SPSS] #linebreak() ] ]