Section 7.3 Paired-Samples T-Test
I mentioned that there is another way to evaluate whether a CBT program impacts participants: to compare antisocial attitudes before and after completing a CBT program. However, it should be noted that randomized experiments such as our example above are considered better than pre- and post-test experimental designs. This is because random assignment can ensure that potential confounding variables are evenly distributed between the treatment and control groups. On the other hand, pre- and post-test experimental designs using the same group of people may not control for all these possible confounding variables, such as time-related effects. Additionally, the participants may be systematically different at baseline when designing pre- and post-test experiments.
Let’s first download the data from the shared Google Drive folder containing the
CBT_dataset_paired.csv data. In this fictitious data, 100 high-risk inmates completed a CBT program, and their survey responses to antisocial attitudes before and after the program were recorded in this dataset. Pre_CBT_Antisocial represents antisocial attitudes before a CBT program, whereas Post_CBT_Antisocial reflects antisocial attitudes after a CBT program. Let’s load the data as we did in the earlier section.
library(readr)
CBT_dataset_paired <- read_csv("CBT_dataset_paired.csv")
View(CBT_dataset_paired)
Subsection 7.3.1 NHST Steps for Paired-Samples T-Test
Subsubsection 7.3.1.1 Step 1: Formulate the Null and Alternative Hypotheses
-
\(H_0\text{:}\) There is no difference in antisocial attitudes between the pre-CBT and post-CBT assessments.
-
\(H_A\text{:}\) There is a difference in antisocial attitudes between the pre-CBT and post-CBT assessments.
Subsubsection 7.3.1.2 Step 2: Calculate the Test Statistic
We will use the
t.test() function but we will use the paired = TRUE argument to conduct a paired t-test.
pairedsampt <- t.test(x = CBT_dataset_paired$Pre_CBT_Antisocial,
y = CBT_dataset_paired$Post_CBT_Antisocial,
paired = TRUE)
pairedsampt
The
t.test() output shows a t-statistic of 5.4437.
Subsubsection 7.3.1.3 Step 3: Determine the Probability (P-Value) of Obtaining a Test Statistic at Least as Extreme as the Observed Value, Assuming No Relationship Exists
The p-value was initially displayed in scientific notation as
3.796e-07. To convert it to a standard numeric format, we will apply the same method used above. The resulting p-value is 0.0000003796, indicating a very low probability of detecting a mean difference of 5.245796 in antisocial attitudes between the pre-CBT and post-CBT assessments.
Subsubsection 7.3.1.4 Steps 4 and 5: Reject or Retain the Null Hypothesis
The t-statistic fell within the rejection region. The probability of this sample coming from a population where the mean antisocial attitudes for pre-CBT and post-CBT assessments are equal is exceedingly low. Therefore, it is probable that the sample is from a population where pre-CBT and post-CBT assessments exhibit different mean antisocial attitudes.
Subsection 7.3.2 Reporting the Results of a Paired-Samples T-Test
Since the results do not show mean scores of antisocial attitudes for pre-CBT and post-CBT assessments, you may want to produce these statistics before reporting the results of a paired-samples t-test.
CBT_dataset_paired %>%
summarize(m_Pre_CBT_Antisocial = mean(x = Pre_CBT_Antisocial),
m_Post_CBT_Antisocial = mean(x = Post_CBT_Antisocial),
sd_Pre_Antisocial = sd(x = Pre_CBT_Antisocial),
sd_Post_Antisocial = sd(x = Post_CBT_Antisocial))
Now you have mean scores and standard deviation values of antisocial attitudes for pre-CBT and post-CBT assessments.
The results from our paired-samples t-test can be presented as follows:
A paired-sample t-test was conducted to compare the antisocial attitudes scores for pre-CBT and post-CBT assessments. There was a significant difference in antisocial attitudes for pre-CBT (\(M = 49.50\)) and post-CBT assessments (\(M = 44.25\text{;}\) \(t(99) = 5.44\text{,}\) \(p < .05\text{,}\) two-tailed).
If you want to see how t-test results are reported in an academic peer-reviewed journal, please see Choi (2020). In this paper, I used a paired-samples t-test to see changes in perceptions of the police among participants before and after watching videos related to the police.
