#set document(title: "12.6 Residuals", author: "Rachel Webb") #set page(width: 8.5in, height: auto, margin: 1in) #import "@preview/cetz:0.5.2" #set text(font: ("STIX Two Text", "Libertinus Serif", "New Computer Modern"), size: 10.5pt, lang: "en") #show math.equation: set text(font: ("STIX Two Math", "New Computer Modern Math")) #set par(justify: true, leading: 0.62em, spacing: 0.9em) #set enum(spacing: 1.1em) // room between list items so tall inline fractions don't collide #set list(spacing: 1.1em) #set table(stroke: 0.5pt + rgb("#c7ccd3")) #let BLUE = rgb("#183B6F") // brand navy — section bars + example/solution labels (white on navy 11.09:1) #let ORANGE = rgb("#A94509") // brand primary-700 — AA-safe deep orange for TEXT (5.93:1 on white; raw brand #F37021 is 2.94:1 and must never carry text) #let RED = rgb("#DC2626") // brand error-600 #let GREEN = rgb("#059669") // brand success-600 (decoration only; small green text uses green-text #007942) #show heading.where(level: 1): it => block(width: 100%, above: 0pt, below: 16pt, fill: gradient.linear(BLUE, rgb("#2C5AA0")), inset: (x: 14pt, y: 12pt), radius: 3pt, text(fill: white, weight: "bold", size: 19pt, it.body)) #show heading.where(level: 2): it => block(width: 100%, above: 18pt, below: 10pt, fill: BLUE, inset: (x: 10pt, y: 6pt), radius: 2pt, text(fill: white, weight: "bold", size: 12pt, it.body)) #show heading.where(level: 3): it => text(fill: ORANGE, weight: "bold", size: 12.5pt, it.body) #show heading.where(level: 4): it => text(fill: BLUE, weight: "bold", size: 10.5pt, it.body) #let examplebox(label, title, body) = block(width: 100%, breakable: true, fill: rgb("#EFF1F5"), stroke: 0.5pt + rgb("#CFDDF0"), radius: 4pt, inset: 10pt, above: 12pt, below: 12pt)[ #block(below: 6pt)[#box(fill: BLUE, inset: (x: 6pt, y: 2pt), radius: 2pt, text(fill: white, weight: "bold", size: 8.5pt, label)) #h(0.4em) #strong[#title]] #body] // rail = decorative left rule (raw brand token); labelcolor = AA-safe label text shade #let notebox(label, rail, labelcolor, tint, body) = block(width: 100%, breakable: true, fill: tint, stroke: (left: 3pt + rail), inset: (left: 10pt, rest: 8pt), radius: (right: 4pt), above: 11pt, below: 11pt)[ #text(fill: labelcolor, weight: "bold", size: 7.5pt, tracking: 0.5pt)[#upper(label)] #linebreak() #body] #let solutionbox(body) = block(above: 4pt, below: 8pt)[ #text(fill: BLUE, weight: "bold", size: 8.5pt)[Solution] #linebreak() #body] #let figph(msg) = block(width: 100%, height: 60pt, fill: rgb("#f6f7f9"), stroke: (paint: rgb("#c7ccd3"), dash: "dashed"), radius: 4pt, inset: 10pt)[ #align(center + horizon, text(fill: rgb("#889"), style: "italic", size: 9pt, msg))] // Standardize inlined figure sizes: measure the natural CeTZ canvas, then scale to a // consistent envelope (aspect-aware; see build_typst.py FIG_* constants). Unlike the // print preamble, dimensions are FLOORED: in an editor a user can trim a figure to a // degenerate 1-D shape (a bare line), and w/h or tw/w would then divide by zero. #let _STD_W = 3.5 #let _WIDE_W = 5.6 #let _MAX_H = 3.4 #let _ASPECT_WIDE = 2.2 #let _UPSCALE_MAX = 1.15 #let stdfig(body) = context { let m = measure(body) let w = calc.max(m.width / 1in, 0.01) let h = calc.max(m.height / 1in, 0.01) let tw = if w / h > _ASPECT_WIDE { _WIDE_W } else { _STD_W } let s = calc.min(tw / w, _MAX_H / h, _UPSCALE_MAX) align(center, box(scale(x: s * 100%, y: s * 100%, reflow: true, body))) } #show figure: set block(breakable: false) #set figure(gap: 8pt) #show figure.caption: set text(size: 8.5pt, fill: rgb("#555")) == 12.6#h(0.6em)Residuals When we overlay the regression equation on a scatterplot, most of the time, the points do not lie on the line itself. The vertical distance between the actual value of #math.equation(block: false, alt: "y")[$y$] and the predicted value of #math.equation(block: false, alt: "y hat")[$hat(y)$] is called the #strong[residual]. The numeric value of the residual is found by subtracting the predicted value of #math.equation(block: false, alt: "y")[$y$] from the actual value of #math.equation(block: false, alt: "y")[$y$]: #math.equation(block: false, alt: "y minus y hat")[$y − hat(y)$]. When we find the line of best fit using least squares regression, this finds the regression equation with the smallest sum of the residuals #math.equation(block: false, alt: "∑ y minus y hat")[$∑ y − hat(y)$]. When your residual is positive, then your data point is above the regression line, when the residual is negative, your data point is below the regression line. If you were to find the residuals for all the sample points and add them up you would get zero. The expected value of the residuals will always be zero. The regression equation is found so that there is just as much distance for the residuals above the line as there is below the line. #notebox("Note", rgb("#8a94a6"), rgb("#556666"), rgb("#f7f8fa"))[ #emph[Measure a residual yourself] Opens the Regression panel on the exam data. Use the fitted line y-hat = 26.742 + 3.216346x to predict at x = 15 (about 74.99), then subtract from the observed grade of 80: the residual is about 5.01 - the length of the yellow arrow in Figure 12-15. Try another point and check whether its residual is positive or negative. - Exam-data regression: find the residual at (15, 80) ] #examplebox("Example 1")[][ Find the residual for the point #math.equation(block: false, alt: "open parenthesis 15 , 80 close parenthesis")[$( 15 , 80 )$] for the exam data. Hours Studied for Exam 20 16 20 18 17 16 15 17 15 16 15 17 16 17 14 Grade on Exam 89 72 93 84 81 75 70 82 69 83 80 83 81 84 76 #solutionbox[ Figure 12-15 is a scatterplot with the regression equation #math.equation(block: false, alt: "y hat equals 26.742 plus 3.216346 x")[$hat(y) = 26.742 + 3.216346 x$] from the exam data. #figure(figph[Scatterplot of grade vs hours studied data, with the regression line of predicted grade and one point marked on the regression line for each x-value present displayed. A double-headed yellow arrow shows the vertical distance between the predicted grade for x=15 and the actual data point of (15, 80).], alt: "Scatterplot of grade vs hours studied data, with the regression line of predicted grade and one point marked on the regression line for each x-value present displayed. A double-headed yellow arrow shows the vertical distance between the predicted grade for x=15 and the actual data point of (15, 80).", caption: [Figure 12-15: Scatterplot showing the linear prediction of grades and the distance from point (15, 80) to the predicted grade for #math.equation(block: false, alt: "x equals 15")[$x = 15$].]) The blue diamonds represent the sample data points. The orange squares are the predicted #math.equation(block: false, alt: "y")[$y$] for each value of #math.equation(block: false, alt: "x")[$x$]. If we connect the orange squares, we get the linear regression equation. The vertical distance between each data point and the regression equation is called the residual. The numeric value can be found by subtracting the observed #math.equation(block: false, alt: "y")[$y$] with its corresponding predicted value, #math.equation(block: false, alt: "y minus y hat")[$y − hat(y)$]. We use #math.equation(block: false, alt: "e sub i")[$e_(i)$] to represent the #math.equation(block: false, alt: "i to the power t h")[$i^(t h)$] residual where #math.equation(block: false, alt: "e sub i equals y sub i minus y hat sub i")[$e_(i) = y_(i) − hat(y)_(i)$]. The residual for the point #math.equation(block: false, alt: "open parenthesis 15 , 80 close parenthesis")[$( 15 , 80 )$] is drawn on the scatterplot vertically as a yellow double-sided arrow to visually show the size of the residual. If you were to predict a student’s exam grade when they studied 15 hours, you would get a predicted grade of #math.equation(block: false, alt: "y hat equals 26.742 plus 3.216346 times 15 equals 74.9865")[$hat(y) = 26.742 + 3.216346 · 15 = 74.9865$]. The residual for the point #math.equation(block: false, alt: "open parenthesis 15 , 80 close parenthesis")[$( 15 , 80 )$] then would be #math.equation(block: false, alt: "y minus y hat equals 80 minus 74.9865 equals 5.0135")[$y − hat(y) = 80 − 74.9865 = 5.0135$]. This is the length of the vertical yellow arrow connecting the point #math.equation(block: false, alt: "open parenthesis 15 , 80 close parenthesis")[$( 15 , 80 )$] to the point #math.equation(block: false, alt: "open parenthesis 15 , 74.9865 close parenthesis")[$( 15 , 74.9865 )$]. ] ] === Standard Error of Estimate The standard deviation of the residuals is called the #strong[standard error of estimate] or #math.equation(block: false, alt: "s")[$s$]. Some texts will use a subscript #math.equation(block: false, alt: "s sub e")[$s_(e)$] or #math.equation(block: false, alt: "s sub e s t")[$s_(e s t)$] to distinguish the different standard deviations from one another. When all of your data points line up in a perfectly straight line, #math.equation(block: false, alt: "s equals 0")[$s = 0$] since none of your points deviate from the regression line. As your data points get more scattered away from a regression line, #math.equation(block: false, alt: "s")[$s$] gets larger. When you are analyzing a regression model, you want #math.equation(block: false, alt: "s")[$s$] to be as small as possible. Standard Error of Estimate #math.equation(block: true, alt: "s sub e s t equals s equals the square root of the fraction ∑ open parenthesis y sub i minus y hat sub i close parenthesis squared over n minus 2 equals the square root of M S E")[$s_(e s t) = s = sqrt(frac(∑ attach(( y_(i) − hat(y)_(i) ), t: 2), n − 2)) = sqrt(M S E)$] The standard error of estimate is the standard deviation of the residuals. The standard error of estimate measures the deviation in the vertical distance from data points to the regression equation. The units of #math.equation(block: false, alt: "s")[$s$] are the same as the units of #math.equation(block: false, alt: "y")[$y$]. #examplebox("Example 2")[][ Use the exam data to find the standard error of estimate. #solutionbox[ To find the #math.equation(block: false, alt: "∑ open parenthesis y sub i minus y hat sub i close parenthesis squared")[$∑ attach(( y_(i) − hat(y)_(i) ), t: 2)$] you would need to find the residual for every data point, square the residuals, and sum them up. This is a lot of math. Recall the regression ANOVA table found earlier. The MSE = 15.4912. #figure(figph[Regression ANOVA table for exam data.], alt: "Regression ANOVA table for exam data.", caption: none) The mean square error is the variance of the residuals, if we take the square root of the MSE we find the standard deviation of the residuals, which is the standard error of estimate. #math.equation(block: true, alt: "s equals the square root of M S E equals the square root of 15.4912 equals 3.9359")[$s = sqrt(M S E) = sqrt(15.4912) = 3.9359$] You can also use the technology to find #math.equation(block: false, alt: "s")[$s$]. #figure(figph[Screenshots of Excel and SPSS regression statistics outputs, and TI-84 and TI-89 linear regression t-test outputs, all showing values of 3.93589 for s, the standard error.], alt: "Screenshots of Excel and SPSS regression statistics outputs, and TI-84 and TI-89 linear regression t-test outputs, all showing values of 3.93589 for s, the standard error.", caption: none) ] ]