#set document(title: "3.3 Measures of Position", author: "OpenStax / XYZ Homework") #set page(width: 8.5in, height: auto, margin: 1in) #import "@preview/cetz:0.5.2" #set text(font: ("STIX Two Text", "Libertinus Serif", "New Computer Modern"), size: 10.5pt, lang: "en") #show math.equation: set text(font: ("STIX Two Math", "New Computer Modern Math")) #set par(justify: true, leading: 0.62em, spacing: 0.9em) #set enum(spacing: 1.1em) // room between list items so tall inline fractions don't collide #set list(spacing: 1.1em) #set table(stroke: 0.5pt + rgb("#c7ccd3")) #let BLUE = rgb("#183B6F") // brand navy — section bars + example/solution labels (white on navy 11.09:1) #let ORANGE = rgb("#A94509") // brand primary-700 — AA-safe deep orange for TEXT (5.93:1 on white; raw brand #F37021 is 2.94:1 and must never carry text) #let RED = rgb("#DC2626") // brand error-600 #let GREEN = rgb("#059669") // brand success-600 (decoration only; small green text uses green-text #007942) #show heading.where(level: 1): it => block(width: 100%, above: 0pt, below: 16pt, fill: gradient.linear(BLUE, rgb("#2C5AA0")), inset: (x: 14pt, y: 12pt), radius: 3pt, text(fill: white, weight: "bold", size: 19pt, it.body)) #show heading.where(level: 2): it => block(width: 100%, above: 18pt, below: 10pt, fill: BLUE, inset: (x: 10pt, y: 6pt), radius: 2pt, text(fill: white, weight: "bold", size: 12pt, it.body)) #show heading.where(level: 3): it => text(fill: ORANGE, weight: "bold", size: 12.5pt, it.body) #show heading.where(level: 4): it => text(fill: BLUE, weight: "bold", size: 10.5pt, it.body) #let examplebox(label, title, body) = block(width: 100%, breakable: true, fill: rgb("#EFF1F5"), stroke: 0.5pt + rgb("#CFDDF0"), radius: 4pt, inset: 10pt, above: 12pt, below: 12pt)[ #block(below: 6pt)[#box(fill: BLUE, inset: (x: 6pt, y: 2pt), radius: 2pt, text(fill: white, weight: "bold", size: 8.5pt, label)) #h(0.4em) #strong[#title]] #body] // rail = decorative left rule (raw brand token); labelcolor = AA-safe label text shade #let notebox(label, rail, labelcolor, tint, body) = block(width: 100%, breakable: true, fill: tint, stroke: (left: 3pt + rail), inset: (left: 10pt, rest: 8pt), radius: (right: 4pt), above: 11pt, below: 11pt)[ #text(fill: labelcolor, weight: "bold", size: 7.5pt, tracking: 0.5pt)[#upper(label)] #linebreak() #body] #let solutionbox(body) = block(above: 4pt, below: 8pt)[ #text(fill: BLUE, weight: "bold", size: 8.5pt)[Solution] #linebreak() #body] #let figph(msg) = block(width: 100%, height: 60pt, fill: rgb("#f6f7f9"), stroke: (paint: rgb("#c7ccd3"), dash: "dashed"), radius: 4pt, inset: 10pt)[ #align(center + horizon, text(fill: rgb("#889"), style: "italic", size: 9pt, msg))] // Standardize inlined figure sizes: measure the natural CeTZ canvas, then scale to a // consistent envelope (aspect-aware; see build_typst.py FIG_* constants). Unlike the // print preamble, dimensions are FLOORED: in an editor a user can trim a figure to a // degenerate 1-D shape (a bare line), and w/h or tw/w would then divide by zero. #let _STD_W = 3.5 #let _WIDE_W = 5.6 #let _MAX_H = 3.4 #let _ASPECT_WIDE = 2.2 #let _UPSCALE_MAX = 1.15 #let stdfig(body) = context { let m = measure(body) let w = calc.max(m.width / 1in, 0.01) let h = calc.max(m.height / 1in, 0.01) let tw = if w / h > _ASPECT_WIDE { _WIDE_W } else { _STD_W } let s = calc.min(tw / w, _MAX_H / h, _UPSCALE_MAX) align(center, box(scale(x: s * 100%, y: s * 100%, reflow: true, body))) } #show figure: set block(breakable: false) #set figure(gap: 8pt) #show figure.caption: set text(size: 8.5pt, fill: rgb("#555")) == 3.3#h(0.6em)Measures of Position === Learning Outcomes By the end of this section, you should be able to: - Define and calculate percentiles, quartiles, and #math.equation(block: false, alt: "z")[$z$]-scores for a dataset. - Use Python to calculate measures of position for a dataset. Common measures of position include percentiles and quartiles as well as #math.equation(block: false, alt: "z")[$z$]-scores, all of which are used to indicate the relative location of a particular datapoint. === Percentiles If a student scores 47 on a biology exam, it is difficult to know if the student did well or poorly compared to the population of all other students taking the exam. #strong[Percentiles] provide a way to assess and compare the distribution of values and the position of a specific data point in relation to the entire dataset by indicating the percentage of data points that fall below it. Specifically, a percentile is a value on a scale of one hundred that indicates the percentage of a distribution that is equal to or below it. Let’s say the student learns they scored in the 90th percentile on the biology exam. This percentile indicates that the student has an exam score higher than 90% of all other students taking the test. This is the same as saying that the student’s score places the student in the top 10% of all students taking the biology test. Thus, this student scoring in the 90th percentile did very well on the exam, even if the actual score was 47. To calculate percentiles, the data must be ordered from smallest to largest and then the ordered data divided into hundredths. If you score in the 80th percentile on an aptitude test, that does not necessarily mean that you scored 80% on the test. It means that 80% of the test scores are the same as or less than your score and the remaining 20% of the scores are the same as or greater than your score. Percentiles are useful for comparing many types of values. For example, a stock market mutual fund might report that the performance for the fund over the past year was in the 80th percentile of all mutual funds in the peer group. This indicates that the fund performed better than 80% of all other funds in the peer group. This also indicates that 20% of the funds performed better than this particular fund. To calculate percentiles for a #emph[specific data value] in a dataset, first order the dataset from smallest to largest and count the number of data values in the dataset. Locate the measurement of interest and count how many data values fall below the measurement. Then the percentile for the measurement is calculated as follows: #math.equation(block: true, alt: "Percentile equals the fraction number of data values below the measurement over total number of data values times 100 % equals the fraction n over N times 100 %")[$"Percentile" #h(0.2em) = frac("number of data values below the measurement", "total number of data values") × 100 % = frac(n, N) × 100 %$] #examplebox("Example 1")[][ The following ordered dataset represents the scores of 15 employees on an aptitude test: 51, 63, 65, 68, 71, 75, 75, 77, 79, 82, 88, 89, 89, 92, 95 Determine the percentile for the employee who scored 88 on the aptitude test. #solutionbox[ There are 15 data values in total, and there are 10 data values below 88. #math.equation(block: true, alt: "Percentile equals the fraction number of data values below the measurement over total number of data values times 100 % equals the fraction 10 over 15 times 100 % equals 67 th percentile")[$"Percentile" #h(0.2em) = frac("number of data values below the measurement", "total number of data values") × 100 % = frac(10, 15) × 100 % = 67 "th percentile"$] ] ] === Quartiles While percentiles separate data into 100 equal parts, #strong[quartiles] separate data into quarters, or four equal parts. To find the quartiles, first find the median, or second quartile. The first quartile, #math.equation(block: false, alt: "Q sub 1")[$Q_(1)$], is the middle value, or median, of the lower half of the data, and the third quartile, #math.equation(block: false, alt: "Q sub 3")[$Q_(3)$], is the middle value of the upper half of the data. Note the following correspondence between quartiles and percentiles: - The first quartile corresponds to the 25th percentile. - The second quartile (which is the median) corresponds to the 50th percentile. - The third quartile corresponds to the 75th percentile. #examplebox("Example 2")[][ Consider the following ordered dataset, which represents the time in seconds for an athlete to complete a 40-yard run: 5.4, 6.0, 6.3, 6.8, 7.1, 7.2, 7.4, 7.5, 7.9, 8.2, 8.7 #solutionbox[ The median, or second quartile, is the middle value in this dataset, which is 7.2. Notice that 50% of the data values are below the median, and 50% of the data values are above the median. The lower half of the data values are 5.4, 6.0, 6.3, 6.8, 7.1. Note that these are the data values below the median. The upper half of the data values are 7.4, 7.5, 7.9, 8.2, 8.7, which are the data values above the median. To find the first quartile, #math.equation(block: false, alt: "Q sub 1")[$Q_(1)$], locate the middle value of the lower half of the data (5.4, 6.0, 6.3, 6.8, 7.1). The middle value of the lower half of the dataset is 6.3. Notice that one-fourth, or 25%, of the data values are below this first quartile, and 75% of the data values are above this first quartile. To find the third quartile, #math.equation(block: false, alt: "Q sub 3")[$Q_(3)$], locate the middle value of the upper half of the data (7.4, 7.5, 7.9, 8.2, 8.7). The middle value of the upper half of the dataset is 7.9. Notice that one-fourth, or 25%, of the data values are above this third quartile, and 75% of the data values are below this third quartile. Thus, the quartiles #math.equation(block: false, alt: "Q sub 1")[$Q_(1)$], #math.equation(block: false, alt: "Q sub 2")[$Q_(2)$], #math.equation(block: false, alt: "Q sub 3")[$Q_(3)$] for this dataset are 6.3, 7.2, 7.9, respectively. ] ] The #strong[interquartile range (IQR)] is a number that indicates the spread of the middle half, or the middle 50%, of the data. It is the difference between the third quartile, #math.equation(block: false, alt: "Q sub 3")[$Q_(3)$], and the first quartile, #math.equation(block: false, alt: "Q sub 1")[$Q_(1)$]. #math.equation(block: true, alt: "IQR equals Q sub 3 minus Q sub 1")[$"IQR" #h(0.2em) = Q_(3) − Q_(1)$] Note that the IQR provides a measure of variability that excludes outliers. In Example 2, the IQR can be calculated as: #math.equation(block: true, alt: "IQR equals Q sub 3 minus Q sub 1 equals 7.9 minus 6.3 equals 1.6")[$"IQR" #h(0.2em) = Q_(3) − Q_(1) = 7.9 − 6.3 = 1.6$] Quartiles and the IQR can be used to flag possible outliers in a dataset. For example, if most employees at a company earn about \$50,000 and the CEO of the company earns \$2.5 million, then we consider the CEO’s salary to be an outlier data value because this salary is significantly different from all the other salaries in the dataset. An outlier data value can also be a value much lower than the other data values in a dataset, so if one employee only makes \$15,000, then this employee’s low salary might also be considered an outlier. To detect outliers, you can use the quartiles and the IQR to calculate a lower and an #strong[upper bound] for outliers. Then any data values below the #strong[lower bound] or above the upper bound will be flagged as outliers. These data values should be further investigated to determine the nature of the outlier condition and whether the data values are valid or not. To calculate the lower and upper bounds for outliers, use the following formulas: #math.equation(block: true, alt: "Lower Bound for Outliers, equals, Q sub 1 minus open parenthesis 1.5 times IQR close parenthesis; Upper Bound for Outliers, equals, Q sub 3 plus open parenthesis 1.5 times IQR close parenthesis")[$"Lower Bound for Outliers" & = & Q_(1) − ( 1.5 · "IQR" #h(0.2em) ) \ "Upper Bound for Outliers" & = & Q_(3) + ( 1.5 · "IQR" #h(0.2em) )$] These formulas typically use 1.5 as a cutoff value to identify outliers in a dataset. #examplebox("Example 3")[][ Calculate the IQR for the following 13 home prices and determine if any of the home prices values are potential outliers. Data values are in US dollars. 389950, 230500, 158000, 479000, 639000, 114950, 5500000, 387000, 659000, 529000, 575000, 488800, 1095000 #solutionbox[ Order the data from smallest to largest. 114950, 158000, 230500, 387000, 389950, 479000, 488800, 529000, 575000, 639000, 659000, 1095000, 5500000 First, determine the median of the dataset. There are 13 data values, so the median is the middle data value, which is 488,800. Next, calculate the #math.equation(block: false, alt: "Q sub 1")[$Q_(1)$] and #math.equation(block: false, alt: "Q sub 3")[$Q_(3)$]. For the first quartile, look at the data values below the median. The two middle data values in this lower half of the data are 230,500 and 387,000. To determine the first quartile, find the mean of these two data values. For the third quartile, look at the data values above the median. The two middle data values in this upper half of the data are 639,000 and 659,000. To determine the third quartile, find the mean of these two data values. #math.equation(block: true, alt: "Q sub 1, equals, the fraction 230,500 plus 387,000 over 2 equals 308,750; Q sub 3, equals, the fraction 639,000 plus 659,000 over 2 equals 649,000")[$Q_(1) & = & frac(230","500 + 387","000, 2) = 308,750 \ Q_(3) & = & frac(639","000 + 659","000, 2) = 649,000$] Now, calculate the interquartile range (IQR): #math.equation(block: true, alt: "IQR equals 649,000 minus 308,750 equals 340,250")[$"IQR" #h(0.2em) = 649,000 − 308,750 = 340,250$] Calculate the value of 1.5 interquartile range (IQR): #math.equation(block: true, alt: "open parenthesis 1.5 close parenthesis open parenthesis IQR close parenthesis equals open parenthesis 1.5 close parenthesis open parenthesis 340,250 close parenthesis equals 510,375")[$( 1.5 ) ( "IQR" #h(0.2em) ) = ( 1.5 ) ( 340,250 ) = 510,375$] Calculate the lower and upper bound for outliers: #math.equation(block: true, alt: "Lower Bound, equals, Q sub 1 minus open parenthesis 1.5 close parenthesis open parenthesis IQR close parenthesis equals 308,750 minus 510,375 equals minus 201,625; Upper Bound, equals, Q sub 3 plus open parenthesis 1.5 close parenthesis open parenthesis IQR close parenthesis equals 649,000 plus 510,375 equals 1,159,375")[$"Lower Bound" & = & Q_(1) − ( 1.5 ) ( "IQR" #h(0.2em) ) = 308,750 − 510,375 = − 201,625 \ "Upper Bound" & = & Q_(3) + ( 1.5 ) ( "IQR" #h(0.2em) ) = 649,000 + 510,375 = 1,159,375$] The lower bound for outliers is −201,625. Of course, no home price is less than −201,625, so no outliers are present for the lower end of the dataset. The upper bound for outliers is 1,159,375. The data value of 5,500,000 is greater than the upper bound of 1,159,375. Therefore, the home price of \$5,500,000 is a potential outlier. This is important because the presence of outliers could potentially indicate data errors or some other anomalies in the dataset that should be investigated. For example, there may have been a data entry error and a home price of \$550,000 was erroneously entered as \$5,500,000. ] ] === #math.equation(block: false, alt: "z")[$z$]-scores The #strong[#math.equation(block: false, alt: "z")[$z$]-score] is a measure of the position of an entry in a dataset that makes use of the mean and standard deviation of the data. It represents the number of standard deviations by which a data value differs from the mean. For example, suppose that in a certain neighborhood, the mean selling price of a home is \$350,000 and the standard deviation is \$40,000. A particular home sells for \$270,000. Based on the selling price of this home, we can calculate the relative standing of this home compared to other home sales in the same neighborhood. The corresponding #math.equation(block: false, alt: "z")[$z$]-score of a measurement considers the given measurement in relation to the mean and standard deviation for the entire population. The formula for a #math.equation(block: false, alt: "z")[$z$]-score calculation is as follows: #math.equation(block: true, alt: "z equals the fraction x minus µ over σ")[$z = frac(x − µ, σ)$] Where: #linebreak() #math.equation(block: false, alt: "x")[$x$] is the measurement #linebreak() #math.equation(block: false, alt: "µ")[$µ$] is the mean #linebreak() #math.equation(block: false, alt: "σ")[$σ$] is the standard deviation Notice that when a measurement is below the mean, the corresponding #math.equation(block: false, alt: "z")[$z$]-score will be a negative value. If the measurement is exactly equal to the mean, the corresponding #math.equation(block: false, alt: "z")[$z$]-score will be zero. If the measurement is above the mean, the corresponding #math.equation(block: false, alt: "z")[$z$]-score will be a positive value. #math.equation(block: false, alt: "z")[$z$]-scores can also be used to identify outliers. Since #math.equation(block: false, alt: "z")[$z$]-scores measure the number of standard deviations from the mean for a data value, a #math.equation(block: false, alt: "z")[$z$]-score of 3 would indicate a data value that is 3 standard deviations above the mean. This would represent a data value that is significantly displaced from the mean, and typically, a #math.equation(block: false, alt: "z")[$z$]-score less than −3 or a #math.equation(block: false, alt: "z")[$z$]-score greater than +3 can be used to flag outliers. #examplebox("Example 4")[][ For the home example in Example 3, the #math.equation(block: false, alt: "x")[$x$] value is the home price of \$270,000, the mean #math.equation(block: false, alt: "µ")[$µ$] is \$350,000, and the standard deviation #math.equation(block: false, alt: "σ")[$σ$] is \$40,000. Calculate the #math.equation(block: false, alt: "z")[$z$]-score. #solutionbox[ The #math.equation(block: false, alt: "z")[$z$]-score can be calculated as follows: #math.equation(block: true, alt: "z equals the fraction x minus µ over σ equals the fraction 270,000 minus 350,000 over 40,000 equals the fraction minus 80,000 over 40,000 equals minus 2")[$z = frac(x − µ, σ) = frac(270","000 − 350","000, 40","000) = frac(− 80","000, 40","000) = − 2$] This #math.equation(block: false, alt: "z")[$z$]-score of −2 indicates that the selling price for this home is 2 standard deviations below the mean, which represents a data value that is significantly below the mean. ] ] === Using Python to Calculate Measures of Position for a Dataset DataFrame.describe() computes different measures of position as well on each column of a dataset. See min, 25%, 50%, 75%, and max in . #figure(figph[A data table summarizing statistics about 966 items in the “movie profit” dataset. The minimum, maximum, and quartile rows are highlighted. The minimum rating is 3.3, the minimum duration is 69, the minimum US gross earnings are \$.01 million, and the minimum worldwide gross earnings are \$176.6 million. The maximum rating is about 9.2, the maximum duration is 238, the maximum US gross earnings are about \$936.7 million, and the maximum worldwide gross earnings are about \$2.847 billion. For the 25th percentile, the rating is about 6.2, the duration is about 101.3, the US gross earnings are about \$90.8 million, and the worldwide gross earnings are about \$223.3 million. For the 50th percentile (median), the rating is about 6.8, the duration is about 116, the US gross earnings are about \$129.2 million, and the worldwide gross earnings are about \$309.3 million. For the 75th percentile, the rating is about 7.4, the duration is about 130, the US gross earnings are about \$187.1 million, and the worldwide gross earnings are about \$472.6 million.], alt: "A data table summarizing statistics about 966 items in the “movie profit” dataset. The minimum, maximum, and quartile rows are highlighted. The minimum rating is 3.3, the minimum duration is 69, the minimum US gross earnings are $.01 million, and the minimum worldwide gross earnings are $176.6 million. The maximum rating is about 9.2, the maximum duration is 238, the maximum US gross earnings are about $936.7 million, and the maximum worldwide gross earnings are about $2.847 billion. For the 25th percentile, the rating is about 6.2, the duration is about 101.3, the US gross earnings are about $90.8 million, and the worldwide gross earnings are about $223.3 million. For the 50th percentile (median), the rating is about 6.8, the duration is about 116, the US gross earnings are about $129.2 million, and the worldwide gross earnings are about $309.3 million. For the 75th percentile, the rating is about 7.4, the duration is about 130, the US gross earnings are about $187.1 million, and the worldwide gross earnings are about $472.6 million.", caption: [The Output of DataFrame.describe() with the Movie Profit Dataset])