8.4 Student-t-test

When we are not given the uncertainty of the measurements, $\sigma$, and the data are insufficient to estimate the uncertainty, then we need to estimate both the “true” value, $\mu$, and the uncertainty. This leads to a wider credible range for the “true” value. We can apply the same testing procedure, by looking at the 95% credible region to see if it includes zero, but this time the posterior distribution we use comes from the so-called Student-$t$ distribution.

1. For $N$ independent observations, we still have the best estimate of the “true” value given by the sample mean

   $$\begin{eqnarray*} \hat{\mu} = \frac{x_{1} + x_{2} + \cdots + x_{N}}{N} \end{eqnarray*}$$

2. The best estimate of the uncertainty of a single measurement is given by the sample standard deviation

   $$\begin{eqnarray*} \hat{\sigma} = S \end{eqnarray*}$$

   where $S$ is the sample standard deviation

   $$\begin{eqnarray*} S^{2}&=&\frac{1}{N-1}\left( (x_{1}-\bar{x})^{2}+\cdots+(x_{N}-\bar{x})^{2}\right) \end{eqnarray*}$$

3. The credible region is determined by the 95% interval of the posterior, Student-$t$ distribution, of the following form

   $$\begin{eqnarray*} {\rm Student}_{{\rm dof}=N-1}(\bar{x}, S/\sqrt{N}) \end{eqnarray*}$$

4. Test to see if the credible range includes zero.
5. If so, then the test passes, and we can be reasonably confident that the parameter is non-zero - that the effect is real.
6. If the test fails, i.e. the credible range does not include zero, then under the model the possibility of a zero-effect cannot be reasonably excluded.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.