Chapter 16 · Advanced Concepts
Resampling
Bootstrap and permutation methods for when formulas run out.
16.1 The bootstrap
Chapter 7 needed a formula for the standard error and Chapter 8 needed a t distribution. The bootstrap needs neither. Treat your one sample as a stand-in for the population. Draw from it with replacement, so some values come up twice and some are missed. Each such resample has its own mean.
\[\text{SE}_{\text{boot}} = \text{SD of the resample means}.\]
Take the middle 95 % of that pile and you have a percentile interval. Watch it land beside the t interval from Chapter 8, which used a formula the bootstrap never touched. The catch is that your sample has to look like the population. Shrink n and both intervals widen, but the bootstrap can only reshuffle what it was given.
Reading: this section is beyond the OpenStax text. See any introduction to bootstrap resampling.
16.2 Permutation tests
Two groups differ by some amount. If the labels mean nothing, then which value wore which label was arbitrary, and any other assignment was equally likely. So try them. Shuffle the labels, recompute the difference, and repeat. The pile that builds up is what chance alone produces, and the p-value is
\[p = \frac{1 + \text{shuffles at least as extreme as the observed gap}}{1 + \text{shuffles tried}}.\]
The two added ones count the observed labelling, which is itself one of the arrangements the null hypothesis allows. No finite run of shuffles can show that p is exactly zero. Notice what does not move. The dots stay exactly where they were, and only their row changes. This is the null hypothesis stated as an action rather than a formula. Its answer lands beside Welch's from Chapter 10, without assuming normal data or equal variances.
Reading: this section is beyond the OpenStax text. See any introduction to permutation tests.