10.5 Computer Examples

from sie import *

data=load_data('data/shoesize.xls')

data.head()

Show expected output

   Index Gender  Size  Height
0      1      F   5.5      60
1      2      F   6.0      60
2      3      F   7.0      60
3      4      F   8.0      60
4      5      F   8.0      60

import random

random.seed(102)
rows = random.sample(data.index, 10)
newdata=data.ix[rows]
data=newdata
data

Show expected output

     Index Gender  Size  Height
60      61      F   7.0      64
251    252      M   9.0      70
69      70      F   8.0      64
290    291      M  11.0      71
247    248      M  12.0      69
156    157      F   9.5      68
231    232      M  10.0      69
17      18      F   6.5      61
216    217      M  10.0      68
252    253      M   9.0      70

plot(data['Height'],data['Size'],'o')
gca().set_xlim([60,72])
gca().set_ylim([4,14])
xlabel('Height [inches]')
ylabel('Shoe Size')

Show expected output

<matplotlib.text.Text at 0x10adae0d0>

Scatter of the nine-person shoe-size subset from the sample code: shoe sizes 6.5 to 12 against heights 61 to 71 inches, trending upward.

result=regression('Size ~ Height',data)

Show expected output

<matplotlib.figure.Figure at 0x10d200710>

Posterior curve for the regression intercept with dashed percentile lines: 1% at -40.27, 50% at -19.26 and 99% at 1.76.

Show expected output

<matplotlib.figure.Figure at 0x10d702610>

Posterior curve for the height coefficient β with dashed percentile lines: 1% at 0.11, 50% at 0.42 and 99% at 0.73; the entire curve sits above zero.

plot(data['Height'],data['Size'],'o')

h=linspace(60,72,10)
plot(h,result['_Predict'](Height=h),'-')

gca().set_xlim([60,72])
gca().set_ylim([4,14])
xlabel('Height [inches]')
ylabel('Shoe Size')

b=result.Intercept.mean()
m=result.Height.mean()

if b>0:
    text(62,12,'$y=%.3f x + %.3f$' % (m,b),fontsize=30)
else:
    text(62,12,'$y=%.3f x %.3f$' % (m,b),fontsize=30)

The shoe-size scatter with the fitted line y=0.422x-19.256 overlaid, running from about (60, 6) to (72, 11.2).

data=load_data('data/sat.csv')

result=regression('total ~ expenditure',data)

Show expected output

<matplotlib.figure.Figure at 0x1109156d0>

Posterior for the SAT regression intercept: a bell centered at 1089.29 with percentile lines from 982.47 (1%) to 1196.12 (99%).

Show expected output

<matplotlib.figure.Figure at 0x110953110>

Posterior for the expenditure coefficient in the SAT fit: a bell centered at -20.89 with percentile lines from -38.53 (1%) to -3.26 (99%) — entirely below zero.

plot(data['expenditure'],data['total'],'o')
xlabel('Expenditure [per pupil, thousands]')
ylabel('SAT Total')
h=linspace(3,10,10)
plot(h,result['_Predict'](expenditure=h),'-')

b=result.Intercept.mean()
m=result.expenditure.mean()

if b>0:
    text(4.5,1125,'$y=%.3f x + %.3f$' % (m,b),fontsize=30)
else:
    text(4.5,1125,'$y=%.3f x %.3f$' % (m,b),fontsize=30)

Scatter of total SAT score (about 840-1110) against per-pupil expenditure in thousands, with the downward fitted line y=-20.892x+1089.294. Spending more appears naively associated with LOWER scores.

result=regression('percent_taking ~ expenditure',data)

Show expected output

<matplotlib.figure.Figure at 0x111cfff10>

Posterior for the intercept of the percent-taking fit: a bell centered at -33.48 with percentile lines from -66.77 (1%) to -0.20 (99%).

Show expected output

<matplotlib.figure.Figure at 0x111867750>

Posterior for the expenditure coefficient in the percent-taking fit: a bell centered at 11.64 with percentile lines from 6.14 (1%) to 17.13 (99%) — entirely above zero.

plot(data['expenditure'],data['percent_taking'],'o')
xlabel('Expenditure [per pupil, thousands]')
ylabel('SAT Total')
h=linspace(3,10,10)
plot(h,result['_Predict'](expenditure=h),'-')

b=result.Intercept.mean()
m=result.expenditure.mean()

if b>0:
    text(4.5,85,'$y=%.3f x + %.3f$' % (m,b),fontsize=30)
else:
    text(4.5,85,'$y=%.3f x %.3f$' % (m,b),fontsize=30)

Scatter of the percentage of students taking the SAT against per-pupil expenditure with the rising fitted line y=11.638x-33.485; higher-spending states have far more of their students take the test. (The plot's y-axis label mistakenly reads 'SAT Total'.)

result=regression('total ~ expenditure + percent_taking',data)

Show expected output

<matplotlib.figure.Figure at 0x1107f5fd0>

Posterior for the intercept of the two-variable SAT model: a bell centered at 993.83 with percentile lines from 941.25 (1%) to 1046.41 (99%).

Show expected output

<matplotlib.figure.Figure at 0x10d70a690>

Posterior for the expenditure coefficient once percent-taking is included: a bell centered at 12.29 with percentile lines from 2.11 (1%) to 22.46 (99%) — now entirely positive.

Show expected output

<matplotlib.figure.Figure at 0x110fbcf90>

Posterior for the percent-taking coefficient in the two-variable SAT model: a bell centered at -2.85 with percentile lines from -3.37 (1%) to -2.33 (99%).

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.