6.8 Uncertainty in the Best Estimates

To quantify the uncertainty in the best estimates, we need a value which represents the width of the distribution. Looking at Figure 6.10 we'd like to provide a quick way of saying that the range of probable values lies somewhere between $θ equals 0.2$ and $θ equals 0.5$ - anything outside of this contributes only a small amount to the probability, or in other words, we are most confident that our best estimate of $θ$ lies between those 0.2 and 0.5. Depending on the application, the symmetry of the distribution, and other practical factors one may see a few potential measures of the width of the distribution.

Inter-Quantile Range The Inter-Quantile Range (ICR) is the range between the 25% and 75% quartiles, and represents 50% of the probability.

In Figure 6.10, the Inter-Quantile Range range is [0.29,0.40].

95% Credible Interval (CI) The 95% Credible Interval (CI) is the range between the 2.5% and 97.5% quantiles, and thus represents 95% of the probability. According to Table 1.1 on page 51, it is “very likely” that our best estimate lies in this range.

In Figure 6.10, the 95% Credible Interval is nearly [0.2,0.5].

Standard Deviation The standard deviation is a measure of the half-width of a distribution, most commonly used specifically with reference to the particular Normal distribution. This will be defined more precisely in Section 7.2 on page 140), and will thus not be defined in general here.

An approximate value for the standard deviation for the Beta distribution is

$\begin{matrix} σ \approx \sqrt{\hat{θ} (1 - \hat{θ}) / N} \end{matrix}$

From Figure 6.10, and using the median as the best estimate, $\hat{θ}$ , we get

$\begin{matrix} σ \approx \sqrt{0.34 (1 - 0.34) / 30} = 0.09 \end{matrix}$

Standard Deviation to Uncertainty To convert this number to an uncertainty, it is a mathematical consequence that about 65% of the area is within 1 value of $σ$ , 95% of the area is within 2 values of $σ$ , and 99% of the area within 3 values.

So, of for the approximate 95% CI for the case shown in Figure 6.10 is

$\begin{matrix} [0.34 - 2 \cdot 0.09, 0.34 + 2 \cdot 0.09] = [0.16, 0.52] \end{matrix}$

a bit more conservative range (larger uncertainty) than is given by the direct method of quantiles, but it much easier to calculate.

Try it in XYZ Crunch

How much the standard-deviation shortcut costs you

Comparing ±2 standard deviations of the Normal(0.34, 0.09) to the book's 5%/95% quantile table mixes two differences. The shortcut uses a 95% central interval (the exact 2.5% and 97.5% points are [0.1636, 0.5164], as you can verify with the first two links). The book's 5%/95% table is a 90% central interval. So part of the gap is simply the wider coverage. The Beta's skew adds more: using the posterior's binomial identity (n = 31, k = 10), the book's rounded 5%-row value p = 0.21 actually gives an area of 0.0451 (link 3), not exactly 5% - the true 5th percentile is 0.2134. Similarly, the 95%-row value p = 0.49 covers 0.9550 (link 4), and the true 95th percentile is 0.4854. Just as in the quartile-table crunch, every printed table entry is a two-decimal-place rounding, not an exact percentile.

Adapted from Statistical Inference for Everyone, by Brian Blais (Bryant University), licensed under CC BY-SA 4.0 (dual-licensed under the GNU FDL 1.2 or later; this adaptation uses the CC BY-SA grant). Changes were made; this adaptation is distributed under the same license. License: CC-BY-SA-4.0.