Assessment on One-Way ANOVA

Recall that null hypothesis tests are of two types: (1) differences between group means and (2) association between variables. In both cases there is a null hypothesis and an alternative hypothesis. In the group means test, the null hypothesis is that the two groups have equal means, and the alternative hypothesis is that the two groups do not have equal means. In the association between variables type of test, the null hypothesis is that the correlation coefficient between the two variables is zero, and the alternative hypothesis is that the correlation coefficient is not zero.

Notice in each case that the hypotheses are mutually exclusive. If the null is false, the alternative must be true. The purpose of null hypothesis statistical tests is generally to show that the null has a low probability of being true (the p value is less than .05) – low enough that the researcher can legitimately claim it is false. The reason this is done is to support the allegation that the alternative hypothesis is true.

In this context you will be studying the details of the first type of test again, with the added capability of comparing the means among more than two group at a time. This is the same type of test of difference between group means. In variations on this model, the groups can actually be the same people under different conditions. The main idea is that several group mean values are being compared. The groups each have an average score or mean on some variable. The null hypothesis is that the difference between all the group means is zero. The alternative hypothesis is that the difference between the means is not zero. Notice that if the null is false, the alternative must be true. It is first instructive to consider some of the details of groups.

One might ask why we would not use multiple t tests in this situation. For instance, with three groups, why would I not compare groups one and two with a t test, then compare groups one and three, and then compare groups two and three?

The answer can be found in our basic probability review. We are concerned with the probability of a TYPE I error (rejecting a true null hypothesis). We generally set an alpha level of .05, which is the probability of making a TYPE I error. Now consider what happens when we do three t tests. There is .05 probability of making a TYPE I error on the first test, .05 probability of the same error on the second test, and .05 probability on the third test. What happens is that these errors are essentially additive, in that the chances of at least one TYPE I error among the three tests much greater than .05. It is like the increased probability of drawing an ace from a deck of cards when we can make multiple draws.

ANOVA allows us do an "overall" test of multiple groups to determine if there are any differences among groups within the set. Notice that ANOVA does not tell us which groups among the three groups are different from each other. The primary test in ANOVA is only to determine if there is a significant difference among the groups somewhere.

You will study the theory and logic of analysis of variance (ANOVA). Recall that a t-test requires a predictor variable that is dichotomous. The advantage of ANOVA over a t-test is that the categorical predictor variable includes 3+ values (groups). Just like a t-test, the outcome variable in ANOVA is quantitative and requires the calculation of group means.

In ANOVA, there are two levels of hypotheses. There is first the overall question of whether all the group means are equal, or if there are some differences among the means somewhere. This is called the omnibus null hypothesis test. The test is designed to show that the probability that the group means are all equal is very low, leading to the researcher being able to legitimately claim there are differences. This is done with the F test. In ANOVA, once the omnibus null hypothesis is rejected, then one may legitimately use special tests, called post hoc tests, to examine each of the pairs of groups in the set to determine which ones differ and which do not. For instance, if an ANOVA is performed for three groups, the omnibus null hypothesis is that the three groups have equal means. If that null is rejected, then the researcher may use special post hoc tests to compare groups 1 & 2, groups 1 & 3, and groups 2 & 3. Each of these post hoc tests are themselves null hypothesis tests, similar to the t tests which were studied previously.

They are designed to control for multiple comparisons, or an inflation of the Type I error rate that is a result of doing many tests with some fixed probability of error on each test. Most are based on the assumption that the omnibus null has been rejected.

The ANOVA, or F-test, relies on predictor variables referred to as factors. A factor is a categorical (nominal) predictor variable. The term "one-way" is applied to an ANOVA with only one factor that is defined by two or more mutually exclusive groups. Technically, an ANOVA can be calculated with only two groups, but the t-test is usually used instead. The one-way ANOVA is usually calculated with three or more groups, which are often referred to as levels of the factor.

If the ANOVA includes multiple factors, it is referred to as a factorial ANOVA. An ANOVA with two factors is referred to as a "two-way" ANOVA; an ANOVA with three factors is referred to as a "three-way" ANOVA, and so on. Factorial ANOVA is studied in Advanced Inferential Statistics. In this course, we will focus on the theory and logic of the one-way ANOVA.

ANOVA is one of the most popular statistics used in psychological research. In nonexperimental designs, the one-way ANOVA compares group means across naturally existing characteristics of groups, such as political affiliation. In experimental designs, the one-way ANOVA compares group means for participants randomly assigned to treatment conditions (for example, high caffeine dose; low caffeine dose; control group).

You may wonder why a one-way ANOVA is necessary. For example, if a factor has four groups ( k = 4), why not just run independent sample t tests for all pairwise comparisons (for example, Group A versus Group B, Group A versus Group C, Group B versus Group C, et cetera)? Warner (2013) points out that a factor with four groups involves six pairwise comparisons. The issue is that conducting multiple pairwise comparisons with the same data leads to inflated risk of a Type I error (incorrectly rejecting a true null hypothesis—getting a false positive). The ANOVA protects the researcher from inflated Type I error by calculating a single omnibus test that assumes all k population means are equal.

Although the advantage of the omnibus test is that it helps protect researchers from inflated Type I error, the limitation is that a significant omnibus test does not specify exactly which group means differ, just that there is a difference "somewhere" among the group means. A researcher therefore relies on either (a) planned contrasts of specific pair wise comparisons determined prior to running the F-test, or, (b) follow-up tests of pair wise comparisons, also referred to as post-hoc tests, to determine exactly which pair wise comparisons are significant. Usually, if planned contrasts are designed correctly, there is no need to perform the omnibus null test, and the overall ANOVA is not necessary.

The null hypothesis of the omnibus test is that all k population means are equal, or H0: µ1 = µ2

=…µk. By contrast, the alternative hypothesis is usually articulated by stipulating that H0 is not true. Keep in mind that this prediction does not imply that all groups must significantly differ from one another on the outcome variable. In fact, for reasons beyond the scope of our present discussion, it is not even strictly necessary that any two groups differ even if the omnibus null is rejected.

The assumptions of ANOVA reflect assumptions of the t-test. ANOVA assumes independence of observations. ANOVA assumes that outcome variable Y is normally distributed. ANOVA assumes that the variance of Y scores is equal across all levels (groups) of the factor. These ANOVA assumptions are checked in the same process used to check assumptions for the t-test discussed earlier in the course—using the Shapiro-Wilk test and the Levene test.

The effect size for a one-way ANOVA is eta squared (η2). It represents the amount of variance in Y that is attributable to group differences. Recall the concept of sum of squares ( SS). Eta squared for the one-way ANOVA is calculated by dividing the sum of squares of between-group differences (SS-between) by the total sums of squares in the model (SS-total), which is reported in SPSS output for the F-test. Eta squared for the one-way ANOVA is interpreted with <.02 as "small," < .06 as "medium," and > .06 as "large."

3