Section 8.4 Appendix: More about ANOVA
Subsection 8.4.1 Terminology for Various Designs
1
This subsection is adapted from David M. Lane. “Analysis of Variance Designs.” Online Statistics Education: A Multimedia Course of Study. https://onlinestatbook.com/2/analysis_of_variance/anova_designs.html
There are many types of experimental designs that can be analyzed by ANOVA. This section discusses many of these designs and defines several key terms used.
Factors and Levels
In describing an ANOVA design, the term factor is a synonym of independent variable. Therefore, in the case study “Smiles and Leniency,” “Type of Smile” is the factor in this experiment. Since four types of smiles were compared, the factor “Type of Smile” has four levels.
An ANOVA conducted on a design in which there is only one factor is called a one-way ANOVA. If an experiment has two factors, then the ANOVA is called a two-way ANOVA. For example, suppose an experiment on the effects of age and gender on reading speed were conducted using three age groups (8 years, 10 years, and 12 years) and the two genders (male and female). The factors would be age and gender. Age would have three levels and gender would have two levels.
Between- and Within-Subjects Factors
In the “Smiles and Leniency” study, the four levels of the factor “Type of Smile” were represented by four separate groups of subjects. When different subjects are used for the levels of a factor, the factor is called a between-subjects factor or a between-subjects variable. The term “between subjects” reflects the fact that comparisons are between different groups of subjects.
In the “ADHD Treatment” study, in which every subject was tested with each of four dosage levels (0, 0.15, 0.30, 0.60 mg/kg) of a drug. Therefore there was only one group of subjects, and comparisons were not between different groups of subjects but between conditions within the same subjects. When the same subjects are used for the levels of a factor, the factor is called a within-subjects factor or a within-subjects variable. Within-subjects variables are sometimes referred to as repeated-measures variables since there are repeated measurements of the same subjects.
Multi-Factor Designs
It is common for designs to have more than one factor. For example, consider a hypothetical study of the effects of age and gender on reading speed in which males and females from the age levels of 8 years, 10 years, and 12 years are tested. There would be a total of six different groups as shown in Table 8.8.
| Group | Gender | Age |
|---|---|---|
| 1 | Female | 8 |
| 2 | Female | 10 |
| 3 | Female | 12 |
| 4 | Male | 8 |
| 5 | Male | 10 |
| 6 | Male | 12 |
This design has two factors: age and gender. Age has three levels and gender has two levels. When all combinations of the levels are included (as they are here), the design is called afactorial design. A concise way of describing this design is as a Gender (2) x Age (3) factorial design where the numbers in parentheses indicate the number of levels. Complex designs frequently have more than two factors and may have combinations of between- and within-subjects factors.
Subsection 8.4.2 Details of One-Factor ANOVA (Between Subjects)
2
This subsection is adapted from David M. Lane. “One-Factor ANOVA (Between Subjects).” Online Statistics Education: A Multimedia Course of Study. https://onlinestatbook.com/2/analysis_of_variance/one-way.html
This section shows how ANOVA can be used to analyze a one-factor between-subjects design.
Analysis of variance is a method for testing differences among means by analyzing variance. The test is based on two estimates of the population variance (\(\sigma^2\)). One estimate is called the mean square error (MSE) and is based on differences among scores within the groups. MSE estimates \(\sigma^2\) regardless of whether the null hypothesis is true (the population means are equal). The second estimate is called the mean square between (MSB) and is based on differences among the sample means. MSB only estimates \(\sigma^2\) if the population means are equal. If the population means are not equal, then MSB estimates a quantity larger than \(\sigma^2\text{.}\) Therefore, if the MSB is much larger than the MSE, then the population means are unlikely to be equal. On the other hand, if the MSB is about the same as MSE, then the data are consistent with the null hypothesis that the population means are equal.
Before proceeding with the calculation of MSE and MSB, it is important to consider the assumptions made by ANOVA:
-
The populations have the same variance. This assumption is called the assumption of homogeneity of variance.
-
The populations are normally distributed.
-
Each value is sampled independently from each other value. This assumption requires that each subject provide only one value. If a subject provides two scores, then the values are not independent; to accommodate such data, one must use within-subjects ANOVA (a type of ANOVA which is easily implemented but which lies beyond the scope of this text).
These assumptions are the same as for a t test of differences between groups (Section 8.1) except that they apply to two or more groups, not just to two groups.
Sample Sizes
As in the main part of the chapter, we will use as our example the “Smiles and Leniency” case study. The first calculations in this section all assume that there is an equal number of observations in each group (unequal sample size calculations are shown later in this appendix). We will refer to the number of observations in each group as n and the total number of observations as N. For these data there are four groups of 34 observations. Therefore, n = 34 and N = 136.
Computing MSE
Recall that the assumption of homogeneity of variance states that the variance within each of the populations (\(\sigma^2\)) is the same. This variance, \(\sigma^2\text{,}\) is the quantity estimated by MSE and is computed as the mean of the sample variances. For these data, the MSE is equal to 2.6489.
Computing MSB
The formula for MSB is based on the fact that the variance of the sampling distribution of the mean is
\begin{equation*}
\sigma^2_\mu = \frac{\sigma^2}{n}
\end{equation*}
where n is the sample size of each group. Rearranging this formula, we have
\begin{equation*}
\sigma^2 = n\sigma^2_\mu.
\end{equation*}
Therefore, if we knew the variance of the sampling distribution of the mean, we could compute \(\sigma^2\) by multiplying it by n. Although we do not know the variance of the sampling distribution of the mean, we can estimate it with the variance of the sample means. For the leniency data, the variance of the four sample means is 0.270. To estimate \(\sigma^2\text{,}\) we multiply the variance of the sample means (0.270) by n (the number of observations in each group, which is 34). We find that MSB = 9.179.
To sum up these steps:
-
Compute the means.
-
Compute the variance of the means.
-
Multiply the variance of the means by n.
Recap
If the population means are equal, then both MSE and MSB are estimates of \(\sigma^2\) and should therefore be about the same. Naturally, they will not be exactly the same since they are just estimates and are based on different aspects of the data: The MSB is computed from the sample means and the MSE is computed from the sample variances.
If the population means are not equal, then MSE will still estimate \(\sigma^2\) because differences in population means do not affect variances. However, differences in population means affect MSB since differences among population means are associated with differences among sample means. It follows that the larger the differences among sample means, the larger the MSB.In short, MSE estimates\(\sigma^2\)whether or not the population means are equal, whereas MSB estimates\(\sigma^2\)only when the population means are equal and estimates a larger quantity when they are not equal.
As shown in Section 8.3.2, we compare the MSE to the MSB by way of an F ratio in order to determine a p-value for the null hypothesis that the population means are all equal.
One-Tailed or Two?
Is the probability value from an F ratio a one-tailed or a two-tailed probability? In the literal sense, it is a one-tailed probability since, as you could see in Figure 8.5earlier in the chapter, the probability is the area in the right-hand tail of the distribution. However, the F ratio is sensitive to any pattern of differences among means. It is, therefore, a test of a two-tailed hypothesis and is best considered a two-tailed test.
Sources of Variation
Why do scores in an experiment differ from one another? Consider the scores of two subjects in the “Smiles and Leniency” study: one from the “False Smile” condition and one from the “Felt Smile” condition. An obvious possible reason that the scores could differ is that the subjects were treated differently (they were in different conditions and saw different stimuli). A second reason is that the two subjects may have differed with regard to their tendency to judge people leniently. A third is that, perhaps, one of the subjects was in a bad mood after receiving a low grade on a test. You can imagine that there are innumerable other reasons why the scores of the two subjects could differ. All of these reasons except the first (subjects were treated differently) are possibilities that were not under experimental investigation and, therefore, all of the differences (variation) due to these possibilities are unexplained. It is traditional to call unexplained variance error even though there is no implication that an error was made. Therefore, the variation in this experiment can be thought of as being either variation due to the condition the subject was in or due to error (the sum total of all reasons the subjects’ scores could differ that were not measured).
One of the important characteristics of ANOVA is that it partitions the variation into its various sources. In ANOVA, the term sum of squares (SSQ) is used to indicate variation. The total variation is defined as the sum of squared differences between each score and the mean of all subjects. The mean of all subjects is called the grand mean and is designated as GM. (When there is an equal number of subjects in each condition, the grand mean is the mean of the condition means.) The total sum of squares is defined as
\begin{equation*}
SSQ_\text{total} = \sum(X - GM)^2
\end{equation*}
which means to take each score, subtract the grand mean from it, square the difference, and then sum up these squared values. For the “Smiles and Leniency” study,\(\text{SSQ}_\text{total} = 377.19\text{.}\)
The sum of squares condition is calculated as shown below.
\begin{equation*}
SSQ_\text{condition} = n \left[ (\bar{X}_1 - GM)^2 + (\bar{X}_2 - GM)^2 + ... +(\bar{X}_k - GM)^2 \right]
\end{equation*}
where n is the number of scores in each group, k is the number of groups, \(\bar{X}_1\) is the mean for Condition 1,\(\bar{X}_2\) is the mean for Condition 2, and \(\bar{X}_k\) is the mean for Condition k. For the Smiles and Leniency study, the values are:
\begin{equation*}
SSQ_\text{condition} = 34 \left[ (5.37-4.83)^2 + (4.91-4.83)^2 + (4.91-4.83)^2 +(4.12-4.83)^2 \right]
\end{equation*}
\begin{equation*}
= 27.5
\end{equation*}
If there are unequal sample sizes, the only change is that the following formula is used for the sum of squares condition:
\begin{equation*}
SSQ_\text{condition} = n_1 (\bar{X}_1 - GM)^2 + n_2 (\bar{X}_2 - GM)^2 + ... + n_k (\bar{X}_k - GM)^2
\end{equation*}
where \(n_i\) is the sample size of the \(i\) th condition.\(\text{SSQ}_\text{total}\) is computed the same way as shown above.
The sum of squares error is the sum of the squared deviations of each score from its group mean. This can be written as
\begin{equation*}
SSQ_\text{error} = \sum(X_{i1} - \bar{X}_1)^2 + \sum(X_{i2} - \bar{X}_2)^2 + ...+ \sum(X_{ik} - \bar{X}_k)^2.
\end{equation*}
where \(X_{i1}\) is the \(i\) th score in group 1 and \(\bar{X}_1\) is the mean for group 1,\(X_{i2}\) is the \(i\) th score in group 2 and \(\bar{X}_2\) is the mean for group 2, etc. For the “Smiles and Leniency” study, the means are: 5.368, 4.912, 4.912, and 4.118. The \(SSQ_\text{error}\) is therefore:
\begin{equation*}
(2.5-5.368)^2 + (5.5-5.368)^2 + ... + (6.5-4.118)^2 = 349.65
\end{equation*}
The sum of squares error can also be computed by subtraction:
\begin{equation*}
SSQ_\text{error} = SSQ_\text{total} - SSQ_\text{condition}
\end{equation*}
\begin{equation*}
SSQ_\text{error} = 377.189 - 27.535 = 349.65
\end{equation*}
Therefore, the total sum of squares of 377.19 can be partitioned into \(SSQ_\text{condition}\)(27.53) and \(SSQ_\text{error}\)(349.66).
Once the sums of squares have been computed, the mean squares (MSB and MSE) can be computed easily. The formulas are:
\begin{equation*}
MSB = \frac{SSQ_\text{condition}}{dfn}
\end{equation*}
where dfn is the degrees of freedom numerator and is equal to k - 1 = 3.
\begin{equation*}
MSB = \frac{27.535}{3} = 9.18
\end{equation*}
which is the same value of MSB obtained previously (except for rounding error). Similarly,
\begin{equation*}
MSE = \frac{SSQ_\text{error}}{dfd}
\end{equation*}
where dfd is the degrees of freedom for the denominator and is equal to N - k.
\begin{equation*}
dfd = 136 - 4 = 132
\end{equation*}
\begin{equation*}
MSE = 349.66/132 = 2.65
\end{equation*}
which is the same as obtained previously (except for rounding error). Note that the dfd is often called the dfe for degrees of freedom error.
As we saw in the main portion of the chapter, SSQ and MSB/MSE can be reported alongside the F statistic and p-value for an ANOVA in a results table (Table 8.7). Since most people conducting ANOVA these days do so with automated computer software, you will likely see a results table along these lines in whatever software you use if you conduct ANOVA yourself.
Formatting Data for Computer Analysis
Most computer programs that compute ANOVAs require your data to be in a specific form. Consider the data in Table 8.9.
| Group 1 | Group 2 | Group 3 |
|---|---|---|
| 3 | 2 | 8 |
| 4 | 4 | 5 |
| 5 | 6 | 5 |
Here there are three groups, each with three observations. To format these data for a computer program, you normally have to use two variables: the first specifies the group the subject is in and the second is the score itself. The reformatted version of the data in Table 8.9 is shown in Table 8.10.
| Group | Y |
|---|---|
| 1 | 3 |
| 1 | 4 |
| 1 | 5 |
| 2 | 2 |
| 2 | 4 |
| 2 | 6 |
| 3 | 8 |
| 3 | 5 |
| 3 | 5 |
