Tukey's Paired Mean Test

Example: For the above example, use Tukey's method for paired comparisons to determine which pairs of the three level means are different.

import statsmodels.stats.multicomp as mc
comp = mc.MultiComparison(x_df['test1_score'], x_df['level'])
post_hoc_res = comp.tukeyhsd(alpha=0.05)
print(post_hoc_res.summary())

Multiple Comparison of Means - Tukey HSD, FWER=0.05 
====================================================
group1 group2 meandiff p-adj  lower    upper  reject
----------------------------------------------------
  high    low -10.8383 0.001 -12.5144 -9.1622   True
  high medium  -4.7717 0.001  -6.4478 -3.0956   True
   low medium   6.0667 0.001   4.3906  7.7428   True
----------------------------------------------------

Interpretation: All pairwise mean comparisons are significant. That is, the mean score1_difference between high and low, high and medium, and low and medium are significantly different.

Two-way ANOVA (Block Design)

Example: The data set below (URL) contains students test1_score, gender, level, section etc.

1. Write a program to read the dataset and display the first rows.
2. Write a program to find test1_score means grouped by level and gender.
3. Write a program to perform ANOVA block design to test if there is sufficient evidence to indicate a difference in the averages of test1_scores among the three levels (low, medium, high) using gender as blocks. Use 10% level of significance.

my_csv_file.csv

import pandas as pd

url='my_csv_file.csv'

x_df = pd.read_csv(url)

x_df.head()  # print the first five rows

x_df.groupby(['level','gender']).agg({'test1_score': ['mean', 'std','size']})

import statsmodels.api as sm
from statsmodels.formula.api import ols
model = ols('test1_score ~ C(level) + C(gender)',data = x_df).fit()
table = sm.stats.anova_lm(model,type=2) # type 2 for non-equal sample sizes
print(table)

              df       sum_sq      mean_sq           F        PR(>F)
C(level)     2.0  3540.854333  1770.427167  116.878417  5.044391e-33
C(gender)    1.0     3.850572     3.850572    0.254203  6.147613e-01
Residual   176.0  2665.977095    15.147597         NaN           NaN

Interpretation: There is significant difference of the test1_score means between the three levles afte blocking by gender. No significant difference in the average test1_scores between the two genders.

Two-Factor ANOVA (Interactions)

Example: The dataset below (URL) contains students test1_score, gender, level, section etc.

1. Write a program to read the dataset and display the first 5 rows.
2. Write a program to display the means grouped by level and interaction
3. Write a program to perform a two-factor ANOVA with level and section. Do the data provide sufficient evidence to indicate an interaction between level and section? Use 10% level of significance.

my_csv_file.csv

import pandas as pd

url='my_csv_file.csv'

x_df = pd.read_csv(url)

x_df.groupby(['level','section']).agg({'test1_score': ['mean', 'std','size']})

import statsmodels.api as sm
from statsmodels.formula.api import ols
model = ols('test1_score ~ C(level)*C(section)',data = x_df).fit()
table = sm.stats.anova_lm(model,type=2) # type 2 for non-equal sample sizes
print(table)

                        df       sum_sq      mean_sq           F        PR(>F)
C(level)               2.0  3494.967554  1747.483777  121.179409  3.062871e-33
C(section)             3.0   125.235378    41.745126    2.894819  3.687668e-02
C(level):C(section)    6.0   133.100312    22.183385    1.538309  1.685520e-01
Residual             167.0  2408.245695    14.420633         NaN           NaN

Interpretation: There is no significant interaction effect between level and section of the average test1_score, because the p-value 1.685520e-01 is not less than the level of significance.10.

The End

Adapted from Python for Introductory Statistics, by Simon Aman (Truman College, City Colleges of Chicago), licensed under CC BY 4.0. Changes were made: reformatted as an accessible XYZ web edition with live in-browser code cells. License: CC-BY-4.0.