Section B.5 Fitting Regression with Categorical Predictors
So far we have explained regression using a numeric input. It turns out we can also use regression with categorical explanatory variables. It is quite straightforward to run it.
There is only one categorical explanatory variable in this dataset, a binary indicator that indicates whether the county is in a Southern State or not. We can also explore this relationship using regression and a regression line. This is how you would express the model:
# We use the as.factor function to tell R that SOUTH is a categorical variable
fit_2 <- lm(HR90 ~ as.factor(SOUTH), data=ncovr)
Notice that there is nothing different in how we ask for the model.
Let’s have a look at the results:
summary(fit_2)
## ## Call: ## lm(formula = HR90 ~ as.factor(SOUTH), data = ncovr) ## ## Residuals: ## Min 1Q Median 3Q Max ## -9.55 -3.34 -1.17 1.93 68.04 ## ## Coefficients: ## Estimate Std. Error t value Pr(>|t|) ## (Intercept) 3.342 0.144 23.2 <2e-16 ## as.factor(SOUTH)1 6.208 0.212 29.2 <2e-16 ## ## (Intercept) *** ## as.factor(SOUTH)1 *** ## --- ## Signif. codes: ## 0 '***' 0.001 '**' 0.01 '*' 0.05 '.' 0.1 ' ' 1 ## ## Residual standard error: 5.88 on 3083 degrees of freedom ## Multiple R-squared: 0.217, Adjusted R-squared: 0.217 ## F-statistic: 854 on 1 and 3083 DF, p-value: <2e-16
As you will see, the output does not look too different. But notice in the print out how the row with the coefficient and other values for our input variable
SOUTH we see a value of 1 after the name of the variable. What does this mean?
It turns out that a linear regression model with just one dichotomous categorical predictor is just the equivalent of a t test. When you only have one predictor, the value of the intercept is the mean value of the reference category and the coefficient for the slope tells you how much higher (if it is positive) or how much lower (if it is negative) is the mean value for the other category in your factor.
The reference category is the one for which R does not print the level next to the name of the variable for which it gives you the regression coefficient. Here we see that the named level is “1”. That’s telling you that the reference category here is “0”. If you look at the codebook, you will see that 1 means the county is in a Southern state. Therefore, the Y intercept in this case is the mean value of homicide rate for the northern counties, whereas the coefficient for the slope is telling you how much higher (since it is a positive value) the mean value is for the southern counties.
mean(ncovr$HR90[ncovr$SOUTH == 0], na.rm=TRUE)
mean(ncovr$HR90[ncovr$SOUTH == 1], na.rm=TRUE) - mean(ncovr$HR90[ncovr$SOUTH == 0], na.rm=TRUE)
## [1] 3.342 ## [1] 6.208
So, to reiterate, for a binary predictor, the coefficient is nothing else than the difference between the mean of the two levels in your factor variable, between the averages in your two groups.
With categorical variables encoded as factors, you always have a situation like this: a reference category and then as many additional coefficients as there are additional levels in your categorical variable. Each of these additional categories is included into the model as dummy variables. Here our categorical variable has two levels, thus we have only one dummy variable. There will always be one fewer dummy variable than the number of levels. The level with no dummy variable, northern counties in this example, is known as the reference category or the baseline.
It turns out then that the regression table is printing out for us a t test of statistical significance for every input in the model. If we look at the table above, this t value is 29.22 and the p value associated with it is near 0. This is indeed considerably lower than the conventional significance level of 0.05. So we could conclude that the probability of obtaining this value if the null hypothesis is true is very low. The \(R^2\) is not too bad either, although lower than we saw when using resource deprivation.
