Beyond One Variable

17.1 Simpson’s paradox

Every group in the data can slope one way while the pooled data slope the other. That is Simpson’s paradox, and nothing about it is a trick of arithmetic. Here the group decides two things at once. It sets where a point sits along \(x\), and how high its cloud sits in \(y\). Push the clouds apart and the pooled line has to run between their centres, which is downhill. Every point inside a cloud still trends uphill.

Press “Ignore the groups” and the colours vanish along with the information. The pooled line was always there. What changes is whether you can see the reason it lies. A lurking variable like this cannot be found in the numbers alone. You have to know it was recorded, and think to condition on it.

Reading: this section is beyond the OpenStax text. See any discussion of Simpson’s paradox and confounding.

17.2 Two predictors

Add a second predictor and the first one’s coefficient changes meaning. Fitting

\[\hat{y} = b_0 + b_1 x_1 + b_2 x_2\]

makes \(b_1\) the effect of \(x_1\) holding \(x_2\) fixed, which is a different quantity from the slope \(x_1\) gets on its own. When the two predictors move together, \(x_1\) fitted alone is credited with \(x_2\)’s effect as well as its own. Set the true \(\beta_1\) negative and \(\beta_2\) positive, then turn the correlation up. The lone slope comes out positive while the two-predictor \(b_1\) recovers the truth. Neither answer is a mistake. They answer different questions, and only one of them was asked.

Reading: this section is beyond the OpenStax text. See any introduction to multiple regression.