The Central Limit Theorem

7.1 CLT for sample means

Pick an ugly parent distribution. Skewed, two-humped and U-shaped all work. Draw a sample of \(n\) values from it and compute their mean. Do it again, and again. The central limit theorem says the pile of sample means turns into a normal curve as \(n\) grows. The shape of the parent does not matter. Its centre is the parent mean \(\mu\), and its spread is \(\sigma/\sqrt{n}\).¹

Start with \(n = 1\), where the pile of “means” is just the parent again. Slide \(n\) to 5, then 30, and keep drawing. The orange curve is not fitted to the data. It is the theorem's prediction, drawn before a single sample is taken. Two humps, a long tail, a U shape. All melt into the same bell. That is why averages of almost anything are normal.

¹ The SD of sample means shown is the sample standard deviation, dividing by \(n - 1\). Reading: OpenStax Introductory Statistics, Section 7.1.

7.2 The standard error

How much does a sample mean wander? The standard error of the mean is the standard deviation of the sampling distribution,

\[\sigma_{\bar{x}} = \frac{\sigma}{\sqrt{n}},\]

so it shrinks with the square root of the sample size. Four times as much data buys only half the spread, and a hundred times as much buys a tenth. Lock a parent and press “Sweep all n”. Each dot is the standard deviation of two thousand simulated sample means at one value of \(n\), and the curve is \(\sigma/\sqrt{n}\). The dots sit on the curve for every parent, because the formula does not care about the shape, only about \(\sigma\). The lower chart shows the pile of means at the chosen \(n\) tightening around \(\mu\).

Reading: OpenStax Introductory Statistics, Sections 7.1, 7.3.

7.3 CLT for sums

Adding instead of averaging changes nothing about the shape. The sum of \(n\) draws is \(n\) times their mean. So it too becomes normal as \(n\) grows, with

\[\text{mean} = n\mu, \qquad \text{SD} = \sigma\sqrt{n}.\]

The sum's spread grows with \(n\), while the mean's spread shrinks, and both facts come from the same \(\sqrt{n}\). Roll ten dice and add them up, many times. Single rolls are flat from 1 to 6, yet the totals pile into a bell around 35. Slide \(n\) up and the axis widens to follow \(n\mu \pm 5\sigma\sqrt{n}\), but the bell keeps its shape. This is the theorem behind why measurement errors, which are sums of many small nudges, are so often normal.

Reading: OpenStax Introductory Statistics, Section 7.2.