Section 3.5 Quick Guide to Interpreting Regression Results 1
This section is written by Nathan Favero.
1
This section is written by Nathan Favero.
Many social science papers report their main results in the form of a regression table. It’s fairly easy to get started interpreting these results using the three S’s:
2
Wheelan, C. (2010.) Introduction to Public Policy. New York: W. W. Norton & Company.
-
Significance: Is the relationship between the two variables strong enough (relative to the precision of the estimate) to be considered statistically reliable?To assess this, check the p-value. For now, you can use the following rule-of-thumb:
3
In other words, can we conclude it is signal rather than noise? See: Fricker Jr, R. D., Burke, K., Han, X., & Woodall, W. H. (2019). Assessing the statistical analyses used in basic and applied social psychology after their p-value ban. The American Statistician, 73(sup1), 374-390.-
If \(p \lt 0.05\text{:}\) the relationship is statistically significant; proceed to evaluating sign and size
-
If \(p \gt 0.05\text{:}\) results are somewhat indeterminate; any association detected between the two variables could easily be caused by coincidence or random “noise” (so you may want to skip evaluating sign and size)
-
-
Sign: Is the relationship positive or negative? Check whether the coefficient has a negative value.
-
Positive coefficient: as the independent variable increases, the dependent variable is predicted to increase
-
Negative coefficient: as the independent variable increases, the dependent variable is predicted to decrease
-
Note about odds ratios: For certain types of (non-linear) regression, odds ratios (which always take on positive values) are sometimes displayed instead of coefficients; with an odds ratio, a value greater than one indicates a positive relationship while a value smaller than one indicates a negative relationship
-
-
Size: How big is the (predictive) effect? This S is often the most difficult to make sense of, and sometimes you may not have enough information to meaningfully evaluate it (e.g., if the units of measurement for a variable are not clearly explained).
-
For linear models: A one-unit increase in the independent variable predicts a \(\hat{\beta_i}\)-unit change in the dependent variable (where \(\hat{\beta_i}\) represents the value of the coefficient estimate)
4
Why \(\hat{\beta_i}\) and not simply \(\beta_i\text{?}\) We use the hat symbol (\(\hat{~}\)) to indicate an estimate or prediction. So by including the hat, we are implying that our coefficient is an estimate of the true population parameter. -
For non-linear models: Interpreting the size of a coefficient is typically more complicated than for a linear model; look for the authors’ explanation of effect size or “magnitude” of association
-
Table 3.5.1 provides an example of regression results in a format similar to what you may encounter in many research publications. Note, however, that many publications do not list exact p-values; instead, they often use one or more asterisks (*) to denote coefficients with p-values smaller than 0.05 (sometimes also flagging p-values falling below various other thresholds).
| Coef. | Std. err. | p-value | |
|---|---|---|---|
| verb_sat | 0.0017 | 0.0010 | 0.10 |
| math_sat | 0.0048 | 0.0012 | 0.00014 |
| (intercept) | -0.91 | 0.42 | 0.033 |
| n | 105 | ||
| \(r^2\) | 0.487 |
Up until now, we have only discussed simple regression, in which we have a single independent variable. But in Table 3.5.1, we find results for a regression where two independent variables—SAT scores on the verbal section (verb_sat) and SAT scores on the math section (math_sat)—are jointly used to predict students’ GPA in computer science classes. It turns out that regression can easily be performed with multiple independent variables, as described in the appendix to this chapter. When we have multiple independent variables, we evaluate each one on its own terms when working through the three S’s. For the results in Table 3.5.1, we can interpret the results as follows:
-
verb_sat: The p-value for this variable (0.10) is greater than 0.05, so this variable is not statistically significant. Therefore, we don’t necessarily need to interpret the sign or size. We might simply say that we could not establish a reliable link between verbal SAT scores and computer science GPA in this model.
5
Note, however, that the absence of evidence is not necessarily evidence of absence. There could very well be a link between verbal SAT scores and computer science GPA—just one that we cannot reliably detect with this analysis (e.g., because our sample is too small to precisely estimate the association). -
math_sat: The p-value (0.00014) is smaller than 0.05, so math_sat is a statistically significant predictor of computer science GPA. The coefficient (0.0048) has a positive sign, so students with higher math SAT scores are predicted to have higher computer science GPAs. When it comes to size, a one-point increase in the math SAT (e.g., getting a 501 instead of a 500) predicts that the computer science GPA will be 0.0048 points higher. That seems very small, but a one-point increase on an SAT is barely noticeable (and not actually possible if scores are always multiples of ten). In this case, we can get a better sense of size if we consider an increase of 100 points in the math SAT, which requires multiplying the coefficient by 100. A 100-point increase in the math SAT (e.g., getting a 600 instead of a 500) predicts a computer science GPA that is 0.48 points higher (\(0.0048 \times 100 = 0.48\)). This is nearly half a grade point higher and would be quite noticeable to most students. Thus, the size of predictive effect now seems reasonably large.
Note that we do not need to apply the three S’s to the intercept (which can also be labeled the “constant”) because it is not a variable. Table 3.5.1 also contains some additional information frequently shown in regression tables: standard errors (which we will learn more about in Chapter 6), the sample size (n=105), and r-squared (a statistic often used to describe how well the regression model overall explains variation in the dependent variable).
Remember that the three S’s are just a starting point. But they should be enough to help you follow along a little easier when reading the results sections of many research publications. If you’ve started working with a statistical software package by now, you can also try running your own regression models and seeing if you can use the three S’s to help you understand the results.
