11.1 One-Way ANOVA

The $z$ - and $t$ -tests can be used to test the equality between two population means $μ sub 1$ and $μ sub 2$ . When we have more than two groups, we would inflate the probability of making a type I error if we were to compare just two at a time and make a conclusion about all the groups together. To account for this P(Type I Error) inflation, we instead will do an analysis of variance (ANOVA) to test the equality between 3 or more population means $μ sub 1, μ sub 2, μ sub 3, …, μ sub k$ .

The F-test (for ANOVA) is a statistical test for testing the equality of $k$ population means.

The one-way ANOVA F-test is a statistical test for testing the equality of $k$ population means from 3 or more groups within one variable or factor. There are many different types of ANOVA; for now, we are going to start with what is commonly referred to as a one-way ANOVA, which has one main effect or factor that is split up into three or more independent treatment levels. In more advanced courses you would learn about dependent groups or two or more factors.

Assumptions:

The populations are normally distributed continuous variables with equal variances.
The observations are independent.

The hypotheses for testing the equality of $k$ population means (ANOVA) are set up with all the means equal to one another in the null hypothesis and at least one mean is different in the alternative hypothesis.

$H_{0} : μ_{1} = μ_{2} = μ_{3} = \dots = μ_{k}$

$H sub 1:$ At least one mean is different.

Even though there is equality in $H sub 0$ , the ANOVA test is testing if the variance between groups is significantly greater than the variance within groups; hence, this will always be set up as a right-tailed test.

We will be using abbreviations for many of the numbers found in this section.

B = Between, W = Within

MS = Mean Square (This is a variance)

MSB = Mean Square (Variance) Between groups.

MSW = Mean Square (Variance) Within groups.

The formula for the F-test statistic is $F = \frac{M S B}{M S W}$ .

Use the F-distribution with degrees of freedom from the between and within groups. The numerator degrees of freedom are equal to the number of groups minus one, that is numerator degrees of freedom are $d f_{B} = k - 1$ . The denominator degrees of freedom are equal to the total of all the sample sizes minus the number of groups, that is denominator degrees of freedom are $d f_{W} = N - k$ .

The sum of squares, degrees of freedom and mean squares are organized in a table called an ANOVA table. Figure 11-1 below is a template for an ANOVA table.

ANOVA table format and equations. — Figure 11-1: ANOVA table template.

Where:

${\bar{χ}}_{i}$ = sample mean from the $i^{t h}$ group

$s_{i}^{2}$ = sample variance from the $i^{t h}$ group

$n_{i}$ = sample size from the $i^{t h}$ group

$k$ = number of groups

$N = n_{1} + n_{2} + \dots + n_{k}$ = sum of the individual sample sizes for groups

Grand mean from all groups = ${\bar{χ}}_{G M} = \frac{\sum χ_{i}}{N}$

Sum of squares between groups = SSB = $∑ n sub i open parenthesis χ bar sub i minus χ bar sub G M close parenthesis squared$

Sum of squares within groups = SSW = $∑ open parenthesis n sub i minus 1 close parenthesis s sub i squared$

Mean squares between groups (or the between-groups variance $s_{B}^{2}$ ) = $M S B = \frac{S S B}{k - 1}$

Mean squares within-group (or error within-groups variance $s_{W}^{2}$ ) = $M S W = \frac{S S W}{N - k}$

$F = \frac{M S B}{M S W}$ is the test statistic.

These calculations can be time-consuming to do by hand, so use technology to find the ANOVA table values, critical value and/or p-value.

Different textbooks and computer software programs use different labels in the ANOVA tables.

The TI-calculators use the word Factor for Between Groups and Error for Within Groups.
Some software packages use Treatment instead of Between groups. You may see different notation depending on which textbook, software or video you are using.
For between groups SSB = SS_B = SSTR = SST = SSF and for within groups SSW = SS_W = SSE.
One thing that is consistent within the ANOVA table is that the between = factor = treatment always appears on the first row of the ANOVA table, and the within = error always is in the second row of the ANOVA table.
The $d f$ column usually is in the second column since we divide the sum of squares by the $d f$ to find the mean squares. However, some software packages will put the $d f$ column before the sum of squares column.
The test statistic is under the column labeled F.
Many software packages will give an extra column for the p-value and some software packages give a critical value too.

Assumption: The population we are sampling from must be approximately normal with equal variances. If these assumptions are not met there are more advanced statistical methods that should be used.

Example 1

An educator wants to see if there is a difference in the average grades given to students for the 4 different instructors who teach intro statistics courses. They randomly choose courses that each of the 4 instructors taught over the last few years and perform an ANOVA test. What would be the correct hypotheses for this test?

Show solution

There are 4 groups (the 4 instructors) so there will be 4 means in the null hypothesis, $H_{0} : μ_{1} = μ_{2} = μ_{3} = m u_{4}$ .

It is tempting to write $μ sub 1 not equal to μ sub 2 not equal to μ sub 3 not equal to m u sub 4$ for the alternative hypothesis, but the test is testing the opposite of all equal is that at least one mean is different. We could for instance have just the mean for group 2 be different and groups 1, 3 and 4 have equal means. If we wanted to write out all the way to have unequal groups, you would have a combination problem with $_{4} C_{3} +_{4} C_{2} +_{4} C_{1}$ ways of getting unequal means. Instead of all of these possibilities, we just write a sentence: “at least one mean is different.”

The hypotheses are:

$H_{0} : μ_{1} = μ_{2} = μ_{3} = m u_{4}$

$H sub 1:$ At least one mean is different.

Example 2

A researcher claims that there is a difference in the average age of assistant professors, associate professors, and full professors at her university. Faculty members are selected randomly and their ages are recorded. Assume faculty ages are normally distributed. Test the claim at the $α$ = 0.01 significance level. The data are listed below.

# Example 11-2: build the whole ANOVA table from the 1-Var Stats summaries
n    <- c(7, 7, 7)            # group sizes
xbar <- c(37, 52, 54)         # sample means
s2   <- c(53, 55.66667, 35)   # sample variances
N <- sum(n); k <- length(n)
gm  <- sum(n * xbar) / N      # grand mean -> 47.66667
SSB <- sum(n * (xbar - gm)^2) # between    -> 1208.66667
SSW <- sum((n - 1) * s2)      # within     -> 862
MSB <- SSB / (k - 1)          # -> 604.3333
MSW <- SSW / (N - k)          # -> 47.8889
Fstat <- MSB / MSW            # -> 12.6195
qf(0.99, k - 1, N - k)        # critical value -> 6.0129
pf(Fstat, k - 1, N - k, lower.tail = FALSE)  # p-value -> 0.0003755
# Edit a mean or a variance above and watch F and the p-value move.

Show solution

The claim is that there is a difference in the average age of assistant professors $open parenthesis μ sub 1 close parenthesis$ , associate professors $open parenthesis μ sub 2 close parenthesis$ , and full professors $open parenthesis μ sub 3 close parenthesis$ at her university.

The correct hypotheses are:

$H_{0} : μ_{1} = μ_{2} = μ_{3}$

$H sub 1:$ At least one mean differs.

We need to compute all of the necessary parts for the ANOVA table and F-test.

Compute the descriptive stats for each group with your calculator using 1–Var Stats L1.

Record the sample size, sample mean, sum of $χ$ , and sample variances. Take the standard deviation $s_{χ}$ and square it to find the variance $s_{χ}^{2}$ for each group.

Assistant Prof	$n sub 1 equals 7$	$χ bar sub 1 equals 37$	$∑ χ sub 1 equals 259$	$s sub 1 squared equals 53$
Associate Prof	$n sub 2 equals 7$	$χ bar sub 2 equals 52$	$∑ χ sub 2 equals 364$	$s sub 2 squared equals 55.6667$
Prof	$n sub 3 equals 7$	$χ bar sub 1 equals 54$	$∑ χ sub 3 equals 378$	$s sub 3 squared equals 35$

Compute the grand mean: $N = n_{1} + n_{2} + n_{3} = 7 + 7 + 7 = 21$ , ${\bar{χ}}_{G M} = \frac{\sum χ_{i}}{N} = \frac{(259 + 364 + 378)}{21} = 47.66667$ .

Compute the sum of squares for between groups: SSB = $\sum n_{i} {({\bar{χ}}_{i} - {\bar{χ}}_{G M})}^{2} = n_{1} {({\bar{χ}}_{1} - {\bar{χ}}_{G M})}^{2} + n_{2} {({\bar{χ}}_{2} - {\bar{χ}}_{G M})}^{2} + n_{3} {({\bar{χ}}_{3} - {\bar{χ}}_{G M})}^{2} = 7 (37 - 47.66667)^{2} + 7 (52 - 47.66667)^{2} + 7 (54 - 47.66667)^{2} = 1208.66667$ .

Compute the sum of squares within groups: SSW = $\sum (n_{i} - 1) s_{i}^{2} = (n_{1} - 1) s_{1}^{2} + (n_{2} - 1) s_{2}^{2} + (n_{3} - 1) s_{3}^{2} = 6 \cdot 53 + 6 \cdot 55.66667 + 6 \cdot 35 = 862$ .

Place the sum of squares into your ANOVA table and add them up to get the total.

Next, find the degrees of freedom: $k equals 3$ since there are 3 groups, so $d f_{B} = k - 1 = 2$ ; $d f_{W} = N - k = 21 - 3 = 18$ .

Add the degrees of freedom to the table to get the total $d f$ .

Compute the mean squares by dividing the sum of squares by their corresponding $d f$ , then add these numbers to the table. $M S B = \frac{S S B}{k - 1} = \frac{1208.66667}{2} = 604.3333 M S W = \frac{S S W}{N - k} = \frac{862}{18} = 47.8889$

The test statistic is the ratio of these two mean squares: $F = \frac{M S B}{M S W} = \frac{604.3333}{47.8889} = 12.6195$ . Add the test statistic to the table under F.

All ANOVA tests are right-tailed tests, so the critical value for a right-tailed F-test is found with the F-distribution. Use $α$ = 0.01 area in the right-tail. The degrees of freedom are $d f sub N equals 2$ , and $d f sub D equals 18$ . The critical value is 6.0129; see the sampling distribution curve in Figure 11-2.

Figure 11-2: Sampling distribution curve with the critical value and calculated F-value shown.

Note that the F-distribution starts at zero, and is skewed to the right. The test statistic of 12.6295 is larger than the critical value of 6.0129 so the decision would be to reject the null hypothesis.

Decision: Reject $H sub 0$ .

Summary: There is enough evidence to support the claim that there is a difference in the average age of assistant professors, associate professors, and full professors at her university.

If we were using the p-value method, then we would use the calculator or computer for a right-tailed test. The p-value = 0.0003755 is less than $α$ = 0.01, which leads to the same decision of rejecting $H sub 0$ that we found using the critical value method.

Alternatively, use technology to compute the ANOVA table and p-value.

TI-84: ANOVA, hypothesis test for the equality of k population means. Note you have to have the actual raw data to do this test on the calculator. Press the [STAT] key and then the [EDIT] function, type the three lists of data into list one, two and three. Press the [STAT] key, arrow over to the [TESTS] menu, arrow down to the option [F:ANOVA(] and press the [ENTER] key. This brings you back to the regular screen where you should now see ANOVA(. Now hit the [2nd] [L1] [,] [2nd] [L2] [,][2nd] [L3][)] keys in that order. You should now see ANOVA(L1,L2,L3); if you had 4 lists you would then have an additional list.

Press the [ENTER] key. The calculator returns the F-test statistic, the p-value, Factor (Between) $d f$ , SS and MS, Error (Within) $d f$ , SS and MS. The last value, Sxp is the square root of the MSE.

TI-89: ANOVA, hypothesis test for the equality of $k$ population means. Go to the [Apps] Stat/List Editor, then type in the data for each group into a separate list (or if you don’t have the raw data, enter the sample size, sample mean and sample variance for group 1 into list1 in that order, repeat for list2, etc.). Press [2^nd] then F6 [Tests], then select C:ANOVA. Select the input method data or stats. Select the number of groups. Press the [ENTER] key to calculate. The calculator returns the F-test statistic, the p-value, Factor (Between) df, SS and MS, Error (Within) df, SS and MS. The last value, Sxp, is the square root of the MSE.

Excel: Type the labels and data into adjacent columns (it is important not to have any blank columns, which would be an additional group counted as zeros). Select the Data tab, then Data Analysis, ANOVA: Single-Factor, then OK.

Next, select all three columns of data at once for the input range. Check the box that says Labels in first row (only select this if you actually selected the labels in the input range). Change your value of alpha and output range will be one cell reference where you want your output to start, see below.

You get the following output:

Excel gives both the p-value and critical value so you can use either method when making your decision, but make sure you are comfortable with both.

Summary

The ANOVA test gives evidence that there is a difference between three or more means. The null hypothesis will always have the means equal to one another versus the alternative hypothesis that at least one mean is different. The F-test results are about the difference in means, but the test is actually testing if the variation between the groups is larger than the variation within the groups. If this between group variation is significantly larger than the within groups then we can say there is a statistically significant difference in the population means. Hence, we are always performing a right-tailed F-test for ANOVA. Make sure to only compare the p-value with $α$ and the test statistic to the critical value.

Adapted from Mostly Harmless Statistics by Rachel Webb (Portland State University), hosted on LibreTexts (stats.libretexts.org) and licensed under CC BY-SA 4.0. Changes were made. License: CC-BY-SA-4.0.