Chapter 9 Hypothesis Testing
We have studied confidence intervals in Chapter 8. We now introduce hypothesis testing, another widely used method for statistical inference. A claim is made about a value or characteristic of the population and then a random sample is used to infer about the plausibility of this claim or hypothesis. For example, in Section 9.2, we use data collected from Spotify to investigate whether metal music is more popular than deep-house music.
Many of the relevant concepts, ideas, and we have already introduced many of the necessary concepts to understand hypothesis testing in Chapter 7 and Chapter 8. We can now expand further on these ideas and provide a general framework for understanding hypothesis tests. By understanding this general framework, you will be able to adapt it to many different scenarios.
The same can be said for confidence intervals. There was one general framework that applies to confidence intervals, and the
infer package was designed around this framework. While the specifics may change slightly for different types of confidence intervals, the general framework stays the same.
We believe that this approach is better for long-term learning than focusing on specific details for specific confidence intervals. We prefer this approach also for hypothesis tests as well, but we will tie the ideas into the traditional theory-based methods as well for completeness.
In Section 9.1 we review confidence intervals and introduce hypothesis tests for one-sample problems; in particular, for the mean \(\mu\text{.}\) We use both theory-based and simulation-based approaches, and we provide some justification why we consider it a better idea to carefully unpack the simulation-based approach for hypothesis testing in the context of two-sample problems. We also show the direct link between confidence intervals and hypothesis tests. In Section 9.2 we introduce the activity that motivates the simulation-based approach for two-sample problems, data collected from Spotify to investigate whether metal music is more popular than deep-house music. In Section 9.3, Section 9.4, and Section 9.5 we explain, conduct, and interpret hypothesis tests, respectively, using the simulation-based approach of permutation. We introduce a case study in Section 9.6, and in Section 9.7 we conclude with a discussion of the theory-based approach for two-sample problems and some additional remarks.
If you’d like more practice or you are curious to see how this framework applies to different scenarios, you can find fully-worked out examples for many common hypothesis tests and their corresponding confidence intervals in the Appendices online. We recommend that you carefully review these examples as they also cover how the general frameworks apply to traditional theory-based methods like the \(t\)-test and normal-theory confidence intervals. You will see there that these traditional methods are just approximations for the computer-based methods we have been focusing on. However, they also require conditions to be met for their results to be valid. Computer-based methods using randomization, simulation, and bootstrapping have much fewer restrictions. Furthermore, they help develop your computational thinking, which is one big reason they are emphasized throughout this book.
Needed packages
If needed, read Section 1.3 for information on how to install and load R packages.
library(tidyverse)
library(moderndive)
library(infer)
library(nycflights23)
library(ggplot2movies)
Recall that loading the
tidyverse package loads many packages that we have encountered earlier. For details refer to Section 1.3. The packages moderndive and infer contain functions and data frames that will be used in this chapter.
