7.2 Analysis of Variance

Analysis of variance, or ANOVA for short, is used to test if the population means of several groups are all equal or if at least one of the groups has a population mean that differs. Thus the null hypothesis and alternative hypothesis for testing k groups is

ANOVA tests the equality of the populations means by comparing variances – thus the name analysis of variance. An important question to answer before covering ANOVA further is a ”Why is ANOVA useful; why not multiple t-tests?” Each hypothesis test has a probability of α of rejecting the null hypothesis even if the null hypothesis is true, a type I error. The latter statement is true for a t-test as well. Thus

1. One t-test has a probability of type I error equal to 1 - (1 - α)¹ = α.
2. Two t-tests have a probability of type I error equal to 1 - (1 - α)².
3. Three t-tests have a probability of type I error equal to 1 - (1 - α)³.
4. Four t-tests have a probability of type I error equal to 1 - (1 - α)⁴.
5. etc.

To test if one or more of the population means differs from any of the others for k groups, you have to do (k 2) t-tests. If k = 5, then you have to test (5 2) = 10 t-tests and the probability of type one error for any of the ten tests with an α = 0.05 equals 1 - (1 - .05)¹⁰ = 0.401, larger than 40%. That is an extremely high probability of making a type one error or any error for that matter.

Using ANOVA to test if the population means of several groups are all equal or if at least one of the groups has a population mean that differs using α, yields the desired probability of a type I error of α.