8.2 z-test

The $z$-test is the simplest test to use, and is perhaps the most common. It is used when we have the following assumptions:

1. We are modeling the data as a true value, $\mu$, with uncertainty
2. We are modeling the as a Normal distribution with known deviation, $\sigma$, as in $\mathrm{N}\mathrm{o}\mathrm{r}\mathrm{m}\mathrm{a}\mathrm{l}(0,\sigma )$.
3. We are assuming independence between the measurements.

The model of the data is

$$\begin{array}{c}\mathrm{d}\mathrm{a}\mathrm{t}\mathrm{a}=\mu +\text{uncertainty with probability Normal(}\mu =0\text{,known }\sigma \text{)}\end{array}$$

where $\mu$ represents the “true” value. The posterior distribution for $\mu$ also follows a Normal distribution, with a smaller uncertainty, $\sigma /\sqrt{N}$ where $N$ is the number of data points.

To use the $z$-test, we perform the following steps:

1. Calculate our best estimate for $\mu$, denoted as $\widehat{\mu }$.
2. Given the known uncertainty, $\sigma$ of a single measurement, determine the range of credible values for $\mu$ within the uncertainty of the estimate for the $N$ observations, $\sigma /\sqrt{N}$.
3. Test to see if the credible range includes zero.
4. If so, then the test passes, and we can be reasonably confident that the parameter is non-zero - that the effect is real.
5. If the test fails, i.e. the credible range does not include zero, then under the model the possibility of a zero-effect cannot be reasonably excluded.

There are several scenarios where we use the $z$-test, each with the same procedure, differing only in the method of estimating the “true” value $\mu$.

1. For $N$ independent observations, $x_1,x_2,\dots ,x_N$ we have the best estimate given by the sample mean, and uncertainty related to the single-measurement deviation, $\sigma$, as

   $$\begin{array}{c}\widehat{\mu }=\frac{x_1+x_2+\cdots +x_N}{N}\pm \sigma /\sqrt{N}\end{array}$$

2. When estimating a proportion, for a large number of events $N$ of which a fraction $f\equiv h/N$ are successful, we have

   $$\begin{array}{c}\widehat{\mu }\approx f\\ \sigma /\sqrt{N}\approx \sqrt{f(1-f)/N}\end{array}$$

3. For smallish data sets, $5<N<30$, where the uncertainty is not known,

   $$\begin{array}{c}\widehat{\mu }\approx \frac{x_1+x_2+\cdots +x_N}{N}\\ \sigma /\sqrt{N}\approx kS/\sqrt{N}\end{array}$$

   where we replace the known $\sigma /\sqrt{N}$ from the previous case with an estimate using the sample standard deviation and an adjustment for small data set parameter $k$,

   $$\begin{array}{c}{S}^2=\frac{1}{N-1}((x_1-\overline{x}{)}^2+\cdots +(x_N-\overline{x}{)}^2)\\ k\equiv 1+\frac{20}{N^2}\end{array}$$

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.