On To Multiple Regression

Section 11.2 On To Multiple Regression

Here’s the Patsy formula for a multiple regression model where mass is a linear function of both flipper length and culmen length.

🔗

Listing 11.2.1. Python Code

🔗

formula = "mass ~ flipper_length + culmen_length"

And here is the result of fitting this model to the data.

🔗

Listing 11.2.2. Python Code

🔗

result = smf.ols(formula, data=adelie).fit()
display_summary(result)

Table 11.2.3.

🔗

	coef	std err	t	[0.025	0.975]
Intercept	-3573.0817	866.739	-4.122	-5285.864	-1860.299
flipper_length	22.7024	4.742	4.787	13.331	32.074
culmen_length	76.3402	11.644	6.556	53.331	99.350

Table 11.2.4.

🔗

R-squared:

0.3949

This model has three coefficients: an intercept and two slopes. The slope associated with flipper length is 22.7, which means we expect a penguin with a longer flipper, by one millimeter, to weight more, by 22.7 grams -- assuming that culmen length is the same. Similarly, we expect a penguin with a longer culmen, by one millimeter, to weight more, by 76.3 grams -- assuming that flipper length is the same.

🔗

The p-values associated with both slopes are small, which means that the contribution of both explanatory variables would be unlikely to happen by chance.

🔗

And the \(R^2\) value is 0.39, higher than the model with only culmen length (0.30) and the model with only flipper length (0.22). So predictions based on both explanatory variables are better than predictions based on either one alone.

🔗

But they are not as much better as we might have hoped. If flipper length reduces MSE by 22% and culmen length reduces it by 30%, why don’t the two of them together reduce it by a total of 52%? The reason is that the explanatory variables are correlated with each other.

🔗

Listing 11.2.5. Python Code

🔗

$ from thinkstats import corrcoef

corrcoef(adelie, "flipper_length", "culmen_length")
np.float64(0.32578471516515944)

A penguin with a longer flipper also has a longer culmen, on average. The explanatory variables contain some information about each other, which means that they contain some of the same information about the response variable. When we add an explanatory variable to the model, the improvement in \(R^2\) reflects only the new information provided by the new variable.

🔗

We see the same pattern if we add culmen depth as a third explanatory variable.

🔗

Listing 11.2.6. Python Code

🔗

formula = "mass ~ flipper_length + culmen_length + culmen_depth"
result = smf.ols(formula, data=adelie).fit()
display_summary(result)

Table 11.2.7.

🔗

	coef	std err	t	[0.025	0.975]
Intercept	-4341.3019	795.117	-5.460	-5912.639	-2769.964
flipper_length	17.4215	4.385	3.973	8.756	26.087
culmen_length	55.3676	11.133	4.973	33.366	77.369
culmen_depth	140.8946	24.216	5.818	93.037	188.752

Table 11.2.8.

🔗

R-squared:

0.5082

This model has four coefficients. All of the p-values are small, which means that the contribution of each explanatory variable would be unlikely to happen by chance. And the \(R^2\) value is about 0.51, somewhat better than the previous model with two explanatory variables (0.39), and better than either model with a single variable (0.22 and 0.30).

🔗

But again, the incremental improvement is smaller than we might have hoped, because culmen depth is correlated with the other two measurements.

🔗

Listing 11.2.9. Python Code

🔗

$ [
    corrcoef(adelie, "culmen_depth", "flipper_length"),
    corrcoef(adelie, "culmen_depth", "culmen_length"),
]
[np.float64(0.30762017939668534), np.float64(0.39149169183587634)]

This example demonstrates a common use of multiple regression, combining multiple explanatory variables to make better predictions. Another common use is to quantify the contribution of one set of variables while controlling for the contribution of another set.

🔗

Prev Top Next