8.4 Student-t-test

When we are not given the uncertainty of the measurements, $\sigma$, and the data are insufficient to estimate the uncertainty, then we need to estimate both the “true” value, $\mu$, and the uncertainty. This leads to a wider credible range for the “true” value. We can apply the same testing procedure, by looking at the 95% credible region to see if it includes zero, but this time the posterior distribution we use comes from the so-called Student-$t$ distribution.

1. For $N$ independent observations, we still have the best estimate of the “true” value given by the sample mean

   $$\begin{array}{c}\widehat{\mu }=\frac{x_1+x_2+\cdots +x_N}{N}\end{array}$$

2. The best estimate of the uncertainty of a single measurement is given by the sample standard deviation

   $$\begin{array}{c}\widehat{\sigma }=S\end{array}$$

   where $S$ is the sample standard deviation

   $$\begin{array}{c}{S}^2=\frac{1}{N-1}((x_1-\overline{x}{)}^2+\cdots +(x_N-\overline{x}{)}^2)\end{array}$$

3. The credible region is determined by the 95% interval of the posterior, Student-$t$ distribution, of the following form

   $$\begin{array}{c}{\mathrm{S}\mathrm{t}\mathrm{u}\mathrm{d}\mathrm{e}\mathrm{n}\mathrm{t}}_{\mathrm{d}\mathrm{o}\mathrm{f}=N-1}(\overline{x},S/\sqrt{N})\end{array}$$

4. Test to see if the credible range includes zero.
5. If so, then the test passes, and we can be reasonably confident that the parameter is non-zero - that the effect is real.
6. If the test fails, i.e. the credible range does not include zero, then under the model the possibility of a zero-effect cannot be reasonably excluded.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.