Confidence Intervals

8.1 What “95% confident” means

A sample mean is a point estimate of \(\mu\), and it is almost never exactly right. A confidence interval wraps it in a margin that says how far off it could reasonably be. When \(\sigma\) is known, the interval is

\[\bar{x} \pm z^{*}\,\frac{\sigma}{\sqrt{n}},\]

where \(z^{*}\) cuts off the chosen tail area, so \(z^{*} = 1.96\) for 95 %. The confidence level is a promise about the method, not about one interval. Draw fifty samples. Each interval either contains \(\mu\) or it does not, and no interval knows which. The orange ones missed. In the long run about 95 % of them capture \(\mu\). Raise the level and the intervals widen. Raise \(n\) and they shrink. Confidence is bought with width or with data.

Reading: OpenStax Introductory Statistics, Section 8.1.

8.2 The t distribution

In practice \(\sigma\) is unknown, so we estimate it with the sample standard deviation \(s\). That estimate is itself noisy, especially for small samples, and the noise costs certainty. William Gosset worked out the price in 1908. Replace the normal with the t distribution, whose degrees of freedom \(df = n - 1\) set how heavy its tails are. The interval becomes \(\bar{x} \pm t^{*}\, s/\sqrt{n}\).

With \(df = 4\) the shaded tails are far fatter than the normal's, and \(t^{*}\) is 2.78 instead of 1.96. Slide \(df\) up and the t curve melts into the normal. By 30 the difference is small. The lower panel takes one real sample and builds both intervals. The t-interval is usually wider. That width is honest, because it admits we had to guess \(\sigma\).

Reading: OpenStax Introductory Statistics, Section 8.2.

8.3 Intervals for proportions

Polls estimate a proportion, the share \(p\) of voters who support something. Ask \(n\) voters, count \(x\) supporters, and the sample proportion is \(\hat{p} = x/n\). Because a count of successes is binomial and nearly normal for large \(n\), the interval has the familiar shape,

\[\hat{p} \pm z^{*}\sqrt{\frac{\hat{p}(1-\hat{p})}{n}}.\]

Poll a hundred voters over and over. The intervals wobble around the true \(p\), and about 95 % of them capture it, exactly as in 8.1. Now drop \(n\) to 10 and push \(p\) toward 0.05. The capture rate falls short of the promise, because the normal approximation needs both \(n p\) and \(n(1-p)\) to be at least 5. Real polls quote a “margin of error” of about three points, which is this formula with \(n \approx 1000\).

Reading: OpenStax Introductory Statistics, Section 8.3.