Skip to main content

Reproducible Research Using R An Open Educational Resource

Christian Martinez

Section 5.8 Post-hoc Tests

To add to our ANOVA and descriptive statistics, we can run what is called Post-hoc Tests. These are meant to dive deeper into the ANOVA model. While an ANOVA model tells us if there is a statistically significant difference, Post-hoc Tests tell us where the difference is. The TukeyHSD command provides the insight.
🔗
TukeyHSD(anova_model)

Tukey multiple comparisons of means
    95% family-wise confidence level

Fit: aov(formula = score ~ method, data = memory)

$method
                          diff        lwr       upr     p adj
Rereading-Flashcards -6.594305 -12.173359 -1.015251 0.0167319
Testing-Flashcards    9.612406   4.033352 15.191460 0.0003304
Testing-Rereading    16.206711  10.627657 21.785765 0.0000000
We have three different rows, with each studying technique compared to each other. The ’diff’ column shows the mean difference between groups (Group 2 − Group 1).
🔗
  • Positive = first group has a higher mean.
    🔗
    🔗
  • Negative = first group has a lower mean.
    🔗
    🔗
Looking here, we can see that testing has a higher mean than both flashcards and rereading when compared against the two. We also look at the p adj, which is just the p-value. All three are statistically significant, specifically the two related to testing.
🔗
We can visualize this difference in mean comparisons using the plot command in base R.
🔗
plot(TukeyHSD(anova_model))
A horizontal dot plot with confidence intervals for three pairwise comparisons: Testing minus Rereading, Testing minus Flashcards, and Rereading minus Flashcards. A vertical dashed line at zero separates significant from non-significant differences. All intervals exclude zero, indicating all comparisons are significant.
Figure 5.8.1. Tukey’s HSD post-hoc comparison plot showing confidence intervals for pairwise mean differences between study techniques. Intervals not crossing zero indicate statistically significant differences following the significant ANOVA.
🔗
Taking all this into consideration:
🔗
  1. Descriptive statistics show that the mean of Testing is higher than the other techniques.
    🔗
    🔗
  2. The ANOVA results show that there is a statistically significant difference between the means of the different techniques.
    🔗
    🔗
  3. The Post-hoc test results show that testing has higher means than Flashcards and Rereading, and that those differences are statistically significant.
    🔗
    🔗
All of this brings us to the conclusion: The Testing studying technique provides the highest memory exam scores.
🔗
🔗
PrevTopNext