Skip to main content

Introductory Statistics

Section A.8 Introduction To Linear Regression

Solution 1. Exercise 1
  1. The residual plot will show randomly distributed residuals around 0. The variance is also approximately constant.
  2. The residuals will show a fan shape, with higher variability for smaller \(x\text{.}\) There will also be many points on the right above the line. There is trouble with the model being fit here.
Solution 2. Exercise 3
  1. Strong relationship, but a straight line would not fit the data.
  2. Strong relationship, and a linear fit would be reasonable.
  3. Weak relationship, and trying a linear fit would be reasonable.
  4. Moderate relationship, but a straight line would not fit the data.
  5. Strong relationship, and a linear fit would be reasonable.
  6. Weak relationship, and trying a linear fit would be reasonable.
Solution 3. Exercise 5
  1. Exam 2 since there is less of a scatter in the plot of final exam grade versus exam 2. Notice that the relationship between Exam 1 and the Final Exam appears to be slightly nonlinear.
  2. Exam 2 and the final are relatively close to each other chronologically, or Exam 2 may be cumulative so has greater similarities in material to the final exam. Answers may vary.
Solution 4. Exercise 7
  1. \(r = -0.7\) \(\to\) (4).
  2. \(r = 0.45\) \(\to\) (3).
  3. \(r = 0.06\) \(\to\) (1).
  4. \(r = 0.92\) \(\to\) (2).
Solution 5. Exercise 9
  1. The relationship is positive, weak, and possibly linear. However, there do appear to be some anomalous observations along the left where several students have the same height that is notably far from the cloud of the other points. Additionally, there are many students who appear not to have driven a car, and they are represented by a set of points along the bottom of the scatterplot.
  2. There is no obvious explanation why simply being tall should lead a person to drive faster. However, one confounding factor is gender. Males tend to be taller than females on average, and personal experiences (anecdotal) may suggest they drive faster. If we were to follow-up on this suspicion, we would find that sociological studies confirm this suspicion.
  3. Males are taller on average and they drive faster. The gender variable is indeed an important confounding variable.
Solution 6. Exercise 11
  1. There is a somewhat weak, positive, possibly linear relationship between the distance traveled and travel time. There is clustering near the lower left corner that we should take special note of.
  2. Changing the units will not change the form, direction or strength of the relationship between the two variables. If longer distances measured in miles are associated with longer travel time measured in minutes, longer distances measured in kilometers will be associated with longer travel time measured in hours.
  3. Changing units doesn’t affect correlation: \(r = 0.636\text{.}\)
Solution 7. Exercise 13
  1. There is a moderate, positive, and linear relationship between shoulder girth and height.
  2. Changing the units, even if just for one of the variables, will not change the form, direction or strength of the relationship between the two variables.
Solution 8. Exercise 15
In each part, we can write the husband ages as a linear function of the wife ages. (a) \(age_{H} = age_{W} + 3\text{.}\) (b) \(age_{H} = age_{W} - 2\text{.}\) (c) \(age_{H} = 2 \times age_{W}\text{.}\) Since the slopes are positive and these are perfect linear relationships, the correlation will be exactly 1 in all three parts. An alternative way to gain insight into this solution is to create a mock data set, e.g. 5 women aged 26, 27, 28, 29, and 30, then find the husband ages for each wife in each part and create a scatterplot.
Solution 9. Exercise 17
Correlation: no units. Intercept: kg. Slope: kg/cm.
Solution 10. Exercise 19
Over-estimate. Since the residual is calculated as \(observed\ -\ predicted\text{,}\) a negative residual means that the predicted value is higher than the observed value.
Solution 11. Exercise 21
  1. There is a positive, very strong, linear association between the number of tourists and spending.
  2. Explanatory: number of tourists (in thousands). Response: spending (in millions of US dollars).
  3. We can predict spending for a given number of tourists using a regression line. This may be useful information for determining how much the country may want to spend in advertising abroad, or to forecast expected revenues from tourism.
  4. Even though the relationship appears linear in the scatterplot, the residual plot actually shows a nonlinear relationship. This is not a contradiction: residual plots can show divergences from linearity that can be difficult to see in a scatterplot. A simple linear model is inadequate for modeling these data. It is also important to consider that these data are observed sequentially, which means there may be a hidden structure not evident in the current plots but that is important to consider.
Solution 12. Exercise 23
  1. First calculate the slope: \(b_1 = R\times s_y/s_x = 0.636 \times 113 / 99 = 0.726\text{.}\) Next, make use of the fact that the regression line passes through the point \((\bar{x},\bar{y})\text{:}\) \(\bar{y} = b_0 + b_1 \times \bar{x}\text{.}\) Plug in \(\bar{x}\text{,}\) \(\bar{y}\text{,}\) and \(b_1\text{,}\) and solve for \(b_0\text{:}\) 51. Solution: \(\widehat{travel time} = 51 + 0.726 \times distance\text{.}\)
  2. \(b_1\text{:}\) For each additional mile in distance, the model predicts an additional 0.726 minutes in travel time. \(b_0\text{:}\) When the distance traveled is 0 miles, the travel time is expected to be 51 minutes. It does not make sense to have a travel distance of 0 miles in this context. Here, the \(y\)-intercept serves only to adjust the height of the line and is meaningless by itself.
  3. \(R^2 = 0.636^2 = 0.40\text{.}\) About 40\% of the variability in travel time is accounted for by the model, i.e. explained by the distance traveled.
  4. \(\widehat{travel time} = 51 + 0.726 \times distance = 51 + 0.726 \times 103 \approx 126\) minutes. (Note: we should be cautious in our predictions with this model since we have not yet evaluated whether it is a well-fit model.).
  5. \(e_i = y_i - \hat{y}_i = 168 - 126 = 42\) minutes. A positive residual means that the model underestimates the travel time.
  6. No, this calculation would require extrapolation.
Solution 13. Exercise 25
  1. \(\widehat{murder} = -29.901 + 2.559 \times poverty\%\text{.}\)
  2. Expected murder rate in metropolitan areas with no poverty is -29. 901 per million. This is obviously not a meaningful value, it just serves to adjust the height of the regression line.
  3. For each additional percentage increase in poverty, we expect murders per million to be higher on average by 2.559.
  4. Poverty level explains 70.52\% of the variability in murder rates in metropolitan areas.
  5. \(\sqrt{0.7052} = 0.8398\text{.}\)
Solution 14. Exercise 27
  1. There is an outlier in the bottom right. Since it is far from the center of the data, it is a point with high leverage. It is also an influential point since, without that observation, the regression line would have a very different slope.
  2. There is an outlier in the bottom right. Since it is far from the center of the data, it is a point with high leverage. However, it does not appear to be affecting the line much, so it is not an influential point.
  3. The observation is in the center of the data (in the x-axis direction), so this point does not have high leverage. This means the point won’t have much effect on the slope of the line and so is not an influential point.
Solution 15. Exercise 29
  1. There is a negative, moderate-to-strong, somewhat linear relationship between percent of families who own their home and the percent of the population living in urban areas in 2010. There is one outlier: a state where 100\% of the population is urban. The variability in the percent of homeownership also increases as we move from left to right in the plot.
  2. The outlier is located in the bottom right corner, horizontally far from the center of the other points, so it is a point with high leverage. It is an influential point since excluding this point from the analysis would greatly affect the slope of the regression line.
Solution 16. Exercise 31
  1. The relationship is positive, moderate-to-strong, and linear. There are a few outliers but no points that appear to be influential.
  2. \(\widehat{weight} = -105.0113 + 1.0176 \times height\text{.}\) Slope: For each additional centimeter in height, the model predicts the average weight to be 1.0176 additional kilograms (about 2.2 pounds). Intercept: People who are 0 centimeters tall are expected to weigh - 105.0113 kilograms. This is obviously not possible. Here, the \(y\)- intercept serves only to adjust the height of the line and is meaningless by itself.
  3. \(H_0\text{:}\) The true slope coefficient of height is zero (\(\beta_1 = 0\)). \(H_A\text{:}\) The true slope coefficient of height is different than zero (\(\beta_1 \neq 0\)). The p-value for the two-sided alternative hypothesis (\(\beta_1 \ne 0\)) is incredibly small, so we reject \(H_0\text{.}\) The data provide convincing evidence that height and weight are positively correlated. The true slope parameter is indeed greater than 0.
  4. \(R^2 = 0.72^2 = 0.52\text{.}\) Approximately 52\% of the variability in weight can be explained by the height of individuals.
Solution 17. Exercise 33
  1. \(H_0\text{:}\) \(\beta_1 = 0\text{.}\) \(H_A\text{:}\) \(\beta_1 \neq 0\text{.}\) The p-value, as reported in the table, is incredibly small and is smaller than 0.05, so we reject \(H_0\text{.}\) The data provide convincing evidence that wives’ and husbands’ heights are positively correlated.
  2. \(\widehat{height}_{W} = 43.5755 + 0.2863 \times height_{H}\text{.}\)
  3. Slope: For each additional inch in husband’s height, the average wife’s height is expected to be an additional 0.2863 inches on average. Intercept: Men who are 0 inches tall are expected to have wives who are, on average, 43.5755 inches tall. The intercept here is meaningless, and it serves only to adjust the height of the line.
  4. The slope is positive, so \(r\) must also be positive. \(r = \sqrt{0.09} = 0.30\text{.}\)
  5. 63.33. Since \(R^2\) is low, the prediction based on this regression model is not very reliable.
  6. No, we should avoid extrapolating.
Solution 18. Exercise 35
  1. \(H_0: \beta_1 = 0; H_A: \beta_1 \ne 0\text{.}\)
  2. The p-value for this test is approximately 0, therefore we reject \(H_0\text{.}\) The data provide convincing evidence that poverty percentage is a significant predictor of murder rate.
  3. \(n = 20, df = 18, T^*_{18} = 2.10\text{;}\) \(2.559 \pm 2.10 \times 0.390 = (1.74, 3.378)\text{;}\) For each percentage point poverty is higher, murder rate is expected to be higher on average by 1.74 to 3.378 per million.
  4. Yes, we rejected \(H_0\) and the confidence interval does not include 0.
Solution 19. Exercise 37
  1. True.
  2. False, correlation is a measure of the linear association between any two numerical variables.
Solution 20. Exercise 39
  1. The point estimate and standard error are \(b_1 = 0.9112\) and \(SE = 0.0259\text{.}\) We can compute a T-score: \(T = (0.9112 - 1)/0.0259 = -3.43\text{.}\) Using \(df=168\text{,}\) the p-value is about 0.001, which is less than \(\alpha = 0.05\text{.}\) That is, the data provide strong evidence that the average difference between husbands’ and wives’ ages has actually changed over time.
  2. \(\widehat{age}_W = 1.5740 + 0.9112 \times age_{H}\text{.}\)
  3. Slope: For each additional year in husband’s age, the model predicts an additional 0.9112 years in wife’s age. This means that wives’ ages tend to be lower for later ages, suggesting the average gap of husband and wife age is larger for older people. Intercept: Men who are 0 years old are expected to have wives who are on average 1.5740 years old. The intercept here is meaningless and serves only to adjust the height of the line.
  4. \(R = \sqrt{0.88} = 0.94\text{.}\) The regression of wives’ ages on husbands’ ages has a positive slope, so the correlation coefficient will be positive.
  5. \(\widehat{age}_W = 1.5740 + 0.9112 \times 55 = 51.69\text{.}\) Since \(R^2\) is pretty high, the prediction based on this regression model is reliable.
  6. No, we shouldn’t use the same model to predict an 85 year old man’s wife’s age. This would require extrapolation. The scatterplot from an earlier exercise shows that husbands in this data set are approximately 20 to 65 years old. The regression model may not be reasonable outside of this range.
Solution 21. Exercise 41
There is an upwards trend. However, the variability is higher for higher calorie counts, and it looks like there might be two clusters of observations above and below the line on the right, so we should be cautious about fitting a linear model to these data.
Solution 22. Exercise 43
  1. \(r = -0.72 \to (2)\text{.}\)
  2. \(r = 0.07 \to (4)\text{.}\)
  3. \(r = 0.86 \to (1)\text{.}\)
  4. \(r = 0.99 \to (3)\text{.}\)