Chapter 17 · Advanced Concepts
Beyond One Variable
How a third variable can reverse a conclusion, and what a coefficient means with two predictors.
17.1 Simpson’s paradox
Every group in the data can slope one way while the pooled data slope the other. That is Simpson’s paradox, and nothing about it is a trick of arithmetic. Here the group decides two things at once. It sets where a point sits along \(x\), and how high its cloud sits in \(y\). Push the clouds apart and the pooled line has to run between their centres, which is downhill. Every point inside a cloud still trends uphill.
Press “Ignore the groups” and the colours vanish along with the information. The pooled line was always there. What changes is whether you can see the reason it lies. A lurking variable like this cannot be found in the numbers alone. You have to know it was recorded, and think to condition on it.
Reading: this section is beyond the OpenStax text. See any discussion of Simpson’s paradox and confounding.
Try it
- Two groups, filled and hollow. Slope in group A and Slope in group B are both positive.
- Press Ignore the groups. One line fits everything, and Slope ignoring groups is negative.
- Slide How far apart the two groups sit to 0. The pooled slope now matches the groups.
- Slide it back to 8. The paradox returns, because the lurking variable pushes one group up and right.
- Slide Slope inside each group negative. Both clouds tilt together and the pooled line follows them.
17.2 Two predictors
Add a second predictor and the first one’s coefficient changes meaning. Fitting
\[\hat{y} = b_0 + b_1 x_1 + b_2 x_2\]
makes \(b_1\) the effect of \(x_1\) holding \(x_2\) fixed, which is a different quantity from the slope \(x_1\) gets on its own. When the two predictors move together, \(x_1\) fitted alone is credited with \(x_2\)’s effect as well as its own. Set the true \(\beta_1\) negative and \(\beta_2\) positive, then turn the correlation up. The lone slope comes out positive while the two-predictor \(b_1\) recovers the truth. Neither answer is a mistake. They answer different questions, and only one of them was asked.
Reading: this section is beyond the OpenStax text. See any introduction to multiple regression.
Try it
- The true effects are β₁ = −0.5 and β₂ = 1.5, and the two predictors correlate at 0.85.
- Read x₁ alone against True β₁. The lone slope is credited with far more than the truth.
- Read x₁ with x₂ in. The two-predictor fit recovers the true −0.5.
- Slide How strongly x₁ and x₂ move together to 0 and press New data. Now the two answers agree.
- Slide True effect of x₂, β₂ to 0. With nothing to confound, x₁ alone is honest again.