#set document(title: "12.4 Simple Linear Regression", author: "Rachel Webb") #set page(width: 8.5in, height: auto, margin: 1in) #import "@preview/cetz:0.5.2" #set text(font: ("STIX Two Text", "Libertinus Serif", "New Computer Modern"), size: 10.5pt, lang: "en") #show math.equation: set text(font: ("STIX Two Math", "New Computer Modern Math")) #set par(justify: true, leading: 0.62em, spacing: 0.9em) #set enum(spacing: 1.1em) // room between list items so tall inline fractions don't collide #set list(spacing: 1.1em) #set table(stroke: 0.5pt + rgb("#c7ccd3")) #let BLUE = rgb("#183B6F") // brand navy — section bars + example/solution labels (white on navy 11.09:1) #let ORANGE = rgb("#A94509") // brand primary-700 — AA-safe deep orange for TEXT (5.93:1 on white; raw brand #F37021 is 2.94:1 and must never carry text) #let RED = rgb("#DC2626") // brand error-600 #let GREEN = rgb("#059669") // brand success-600 (decoration only; small green text uses green-text #007942) #show heading.where(level: 1): it => block(width: 100%, above: 0pt, below: 16pt, fill: gradient.linear(BLUE, rgb("#2C5AA0")), inset: (x: 14pt, y: 12pt), radius: 3pt, text(fill: white, weight: "bold", size: 19pt, it.body)) #show heading.where(level: 2): it => block(width: 100%, above: 18pt, below: 10pt, fill: BLUE, inset: (x: 10pt, y: 6pt), radius: 2pt, text(fill: white, weight: "bold", size: 12pt, it.body)) #show heading.where(level: 3): it => text(fill: ORANGE, weight: "bold", size: 12.5pt, it.body) #show heading.where(level: 4): it => text(fill: BLUE, weight: "bold", size: 10.5pt, it.body) #let examplebox(label, title, body) = block(width: 100%, breakable: true, fill: rgb("#EFF1F5"), stroke: 0.5pt + rgb("#CFDDF0"), radius: 4pt, inset: 10pt, above: 12pt, below: 12pt)[ #block(below: 6pt)[#box(fill: BLUE, inset: (x: 6pt, y: 2pt), radius: 2pt, text(fill: white, weight: "bold", size: 8.5pt, label)) #h(0.4em) #strong[#title]] #body] // rail = decorative left rule (raw brand token); labelcolor = AA-safe label text shade #let notebox(label, rail, labelcolor, tint, body) = block(width: 100%, breakable: true, fill: tint, stroke: (left: 3pt + rail), inset: (left: 10pt, rest: 8pt), radius: (right: 4pt), above: 11pt, below: 11pt)[ #text(fill: labelcolor, weight: "bold", size: 7.5pt, tracking: 0.5pt)[#upper(label)] #linebreak() #body] #let solutionbox(body) = block(above: 4pt, below: 8pt)[ #text(fill: BLUE, weight: "bold", size: 8.5pt)[Solution] #linebreak() #body] #let figph(msg) = block(width: 100%, height: 60pt, fill: rgb("#f6f7f9"), stroke: (paint: rgb("#c7ccd3"), dash: "dashed"), radius: 4pt, inset: 10pt)[ #align(center + horizon, text(fill: rgb("#889"), style: "italic", size: 9pt, msg))] // Standardize inlined figure sizes: measure the natural CeTZ canvas, then scale to a // consistent envelope (aspect-aware; see build_typst.py FIG_* constants). Unlike the // print preamble, dimensions are FLOORED: in an editor a user can trim a figure to a // degenerate 1-D shape (a bare line), and w/h or tw/w would then divide by zero. #let _STD_W = 3.5 #let _WIDE_W = 5.6 #let _MAX_H = 3.4 #let _ASPECT_WIDE = 2.2 #let _UPSCALE_MAX = 1.15 #let stdfig(body) = context { let m = measure(body) let w = calc.max(m.width / 1in, 0.01) let h = calc.max(m.height / 1in, 0.01) let tw = if w / h > _ASPECT_WIDE { _WIDE_W } else { _STD_W } let s = calc.min(tw / w, _MAX_H / h, _UPSCALE_MAX) align(center, box(scale(x: s * 100%, y: s * 100%, reflow: true, body))) } #show figure: set block(breakable: false) #set figure(gap: 8pt) #show figure.caption: set text(size: 8.5pt, fill: rgb("#555")) == 12.4#h(0.6em)Simple Linear Regression A linear regression is a straight line that describes how the values of a response variable #math.equation(block: false, alt: "y")[$y$] change as the predictor variable #math.equation(block: false, alt: "x")[$x$] changes. The equation of a line, relating #math.equation(block: false, alt: "x")[$x$] to #math.equation(block: false, alt: "y")[$y$] uses the slope-intercept form of a line, but with different letters than what you may be used to in a math class. We let #math.equation(block: false, alt: "b sub 0")[$b_(0)$] represent the sample #math.equation(block: false, alt: "y")[$y$]-intercept (the value of #math.equation(block: false, alt: "y")[$y$] when #math.equation(block: false, alt: "x equals 0")[$x = 0$]), #math.equation(block: false, alt: "b sub 1")[$b_(1)$] the sample slope (rise over run), and #math.equation(block: false, alt: "y hat")[$hat(y)$] the predicted value of #math.equation(block: false, alt: "y")[$y$] for a specific value of #math.equation(block: false, alt: "x")[$x$]. The equation is written as #math.equation(block: false, alt: "y hat equals b sub 0 plus b sub 1 x")[$hat(y) = b_(0) + b_(1) x$]. Some textbooks and the TI calculators use the letter #math.equation(block: false, alt: "a")[$a$] to represent the #math.equation(block: false, alt: "y")[$y$]-intercept and #math.equation(block: false, alt: "b")[$b$] to represent the slope, and the equation is written as #math.equation(block: false, alt: "y hat equals a plus b x")[$hat(y) = a + b x$]. These letters are just symbols representing the placeholders for the numeric values for the #math.equation(block: false, alt: "y")[$y$]-intercept and slope. If we were to fit the best line that was closest to all the points on the scatterplot we would get what we call the “line of best fit,” also known as the “regression equation” or “least squares regression line.” Figure 12-9 is a scatterplot with just five points. #figure(figph[A scatterplot containing 5 points.], alt: "A scatterplot containing 5 points.", caption: [Figure 12-9: A scatterplot.]) Figure 12-10 shows the #strong[least-squares regression line] of #math.equation(block: false, alt: "y")[$y$] on #math.equation(block: false, alt: "x")[$x$], which is the line that minimizes the squared vertical distance from all of the data. If we were to fit the line that best fits through the points, we would get the line pictured below. #figure(figph[The scatterplot, containing an additional line that passes roughly an equal distance from each of the points.], alt: "The scatterplot, containing an additional line that passes roughly an equal distance from each of the points.", caption: [Figure 12-10: Scatterplot with least-squares regression line.]) What we want to look for is the minimum of the squared vertical distance between each point and the regression equation, called a residual. This is where the name of the least squares regression line comes from. Figure 12-11 shows the squared residuals. #figure(figph[Scatterplot with regression line and squared residuals. Residuals are represented by each point above the line being located at the top left corner of a square whose lower left corner rests on the line, and each point below the line being at the bottom left corner of a square whose upper left corner rests on the line.], alt: "Scatterplot with regression line and squared residuals. Residuals are represented by each point above the line being located at the top left corner of a square whose lower left corner rests on the line, and each point below the line being at the bottom left corner of a square whose upper left corner rests on the line.", caption: [Figure 12-11: Scatterplot with least-squares regression line and squared residuals.]) To find the slope and #math.equation(block: false, alt: "y")[$y$]-intercept for the equation of the least-squares regression line #math.equation(block: false, alt: "y hat equals b sub 0 plus b sub 1 x")[$hat(y) = b_(0) + b_(1) x$] we use the following formulas: slope #math.equation(block: false, alt: "equals b sub 1 equals the fraction S S sub x y over S S sub x x")[$= b_(1) = frac(S S_(x y), S S_(x x))$], #math.equation(block: false, alt: "y")[$y$]-intercept: #math.equation(block: false, alt: "b sub 0 equals y bar minus b sub 1 x bar")[$b_(0) = overline(y) − b_(1) overline(x)$]. To compute the least squares regression line, you will need to first find the slope. Then substitute the slope into the following equation of the #math.equation(block: false, alt: "y")[$y$]-intercept: #math.equation(block: false, alt: "b sub 0 equals y bar minus b sub 1 x bar")[$b_(0) = overline(y) − b_(1) overline(x)$], where #math.equation(block: false, alt: "x bar")[$overline(x)$] = the sample mean of the #math.equation(block: false, alt: "x")[$x$]’s and #math.equation(block: false, alt: "y bar")[$overline(y)$] = the sample mean of the #math.equation(block: false, alt: "y")[$y$]’s. Once we find the equation for the regression line, we can use it to estimate the response variable #math.equation(block: false, alt: "y")[$y$] for a specific value of the predictor variable #math.equation(block: false, alt: "x")[$x$]. Note: we would only want to use the regression equation for prediction if we reject #math.equation(block: false, alt: "H sub 0")[$H_(0)$] and find that there is a significant correlation between #math.equation(block: false, alt: "x")[$x$] and #math.equation(block: false, alt: "y")[$y$]. Alternatively, we could start with the regression equation and then test to see if the slope is significantly different from zero. #examplebox("Example 1")[][ Use the following data to find the line of best fit. #notebox("Note", rgb("#8a94a6"), rgb("#556666"), rgb("#f7f8fa"))[ #emph[Read the least-squares line from the panel] Opens the Regression panel with the exam data preloaded. Read the slope b1 = 3.2163 and intercept b0 = 26.742 and write y-hat = 26.742 + 3.216346x - no VARS/RegEQ workflow needed. Then plug in x = 19 by hand to match the calculator's traced value of 87.8526. - y-hat = 26.742 + 3.216346x (r = 0.8254) ] Hours Studied for Exam 20 16 20 18 17 16 15 17 15 16 15 17 16 17 14 Grade on Exam 89 72 93 84 81 75 70 82 69 83 80 83 81 84 76 #solutionbox[ Start with finding the 2-Var Stats and sum of squares as shown in the steps for correlation. #math.equation(block: true, alt: "S S sub x x equals open parenthesis n minus 1 close parenthesis s sub x squared equals open parenthesis 15 minus 1 close parenthesis 1.723783215 squared equals 41.6")[$S S_(x x) = ( n − 1 ) s_(x)^(2) = ( 15 − 1 ) 1.723783215^(2) = 41.6$] #math.equation(block: true, alt: "S S sub y y equals open parenthesis n minus 1 close parenthesis s sub y squared equals open parenthesis 15 minus 1 close parenthesis 6.717425811 squared equals 631.7333")[$S S_(y y) = ( n − 1 ) s_(y)^(2) = ( 15 − 1 ) 6.717425811^(2) = 631.7333$] #math.equation(block: true, alt: "S S sub x y equals ∑ open parenthesis x y close parenthesis minus n times x bar times y bar equals 20087 – open parenthesis 15 times 16.6 times 80.133333 close parenthesis equals 133.8")[$S S_(x y) = ∑ ( x y ) − n · overline(x) · overline(y) = 20087 – ( 15 · 16.6 · 80.133333 ) = 133.8$] Calculate the slope: #math.equation(block: false, alt: "b sub 1 equals the fraction S S sub x y over S S sub x x equals the fraction 133.8 over 41.6 equals 3.216346")[$b_(1) = frac(S S_(x y), S S_(x x)) = frac(133.8, 41.6) = 3.216346$]. Calculate the #math.equation(block: false, alt: "y")[$y$]-intercept: #math.equation(block: false, alt: "b sub 0 equals y bar minus b sub 1 times x bar equals 80.133333 minus 3.216346 times 16.6 equals 26.742")[$b_(0) = overline(y) − b_(1) · overline(x) = 80.133333 − 3.216346 · 16.6 = 26.742$]. Put these numbers back into the regression equation and write your answer as: #math.equation(block: false, alt: "y hat equals 26.742 plus 3.216346 x")[$hat(y) = 26.742 + 3.216346 x$]. Interpreting the #math.equation(block: false, alt: "y")[$y$]-intercept coefficient: When #math.equation(block: false, alt: "x equals 0")[$x = 0$], note that #math.equation(block: false, alt: "y hat equals 26.742")[$hat(y) = 26.742$]. This means that we would expect a failing midterm score of 26.742 for students who had studied zero hours. Interpreting the slope coefficient: For each additional hour studied for the exam, we would expect an increase in the midterm grade of 3.2163 points. In general, when interpreting the slope coefficient, for each additional 1 unit increase in #math.equation(block: false, alt: "x")[$x$], the predicted #math.equation(block: false, alt: "y hat")[$hat(y)$] value will change by #math.equation(block: false, alt: "b sub 1")[$b_(1)$] units. === Adding the Regression Line to the Scatterplot #strong[TI-84:] Make a scatterplot using the directions from the previous section. Turn your STAT scatter plot on. Press \[Y=\] and clear any equations that are in the #math.equation(block: false, alt: "y")[$y$]-editor. Into Y1, enter the least-squares regression equation manually as found above. Or, press the VARS key, go to option 5: Statistics, arrow over to EQ for equation, then choose the first option RegEQ. This will bring the equation over to the Y= menu without rounding error. Press \[GRAPH\]. You can press \[TRACE\] and use the arrow keys to scroll left or right. Pressing up or down on the arrow keys will change between tracing the scatterplot and the regression line. You can use the regression line to predict values of the response variable for a given value of the explanatory variable. While tracing the regression line type the value of the explanatory variable and press \[ENTER\]. For example, for #math.equation(block: false, alt: "x equals 19")[$x = 19$] the value of #math.equation(block: false, alt: "y hat equals 87.8526")[$hat(y) = 87.8526$]. #figure(figph[Setting up the scatterplot with regression line in a TI-84 calculator.], alt: "Setting up the scatterplot with regression line in a TI-84 calculator.", caption: none) #figure(figph[Tracing the scatterplot and the regression line with a TI-84 calculator.], alt: "Tracing the scatterplot and the regression line with a TI-84 calculator.", caption: none) #strong[TI-89:] Make a scatterplot and find the regression line using the directions in the previous section. If you press \[♦\] then \[F1\] (Y=) you will notice the regression equation has been stored into y1 in the y-editor. Press \[F2\] #strong[Trace] and use the left and right arrow keys to trace along the plot. Use the up and down arrow keys to toggle between the regression line and the scatterplot. You can use the regression line to predict values of the response variable for a given value of the explanatory variable. While tracing the regression line type the value of the explanatory variable and press \[ENTER\]. For example, for #math.equation(block: false, alt: "x equals 19")[$x = 19$] the value of #math.equation(block: false, alt: "y equals 87.8526")[$y = 87.8526$]. #figure(figph[Tracing a scatterplot and regression line using a TI-89 calculator.], alt: "Tracing a scatterplot and regression line using a TI-89 calculator.", caption: none) ] ]