Chapter 06: Repeated-measures ANOVA We can also use analysis of variance (ANOVA) in order to test for differences between three or more groups (or samples), when the same participants/ subjects are be used across conditions of an independent variable (or, 1 sample for 3+ measures) By using the repeated-measures (or dependent-measures) ANOVA, individual differences may be accounted for in the error term, as we can directly measure how manipulations of the independent measure may cause changes in behavioral scores across all conditions. Recall that the F-statistic is a ratio that evaluates sources of variance, where total variance is due to treatment effects (between groups) and random factors (specifically individual differences and experimental error). In an independent ANOVA, a ratio of variance between-groups (numerator) relative to variance within groups (denominator) seeks to evaluate how much of the total variance is due to treatment effects (variance between each sample). Thus, measurements of variance due to random factors is essentially divided out. However, within-groups measures obscures how much of the variance from random factors is due to experimental error and how much is due to individual differences. By creating a measure of individual differences across groups, the denominator needs only to only account for error variance due to experimental error. Therefore, an advantage of the repeated -measures ANOVA is the additional power provided from being able to measure the impact of the manipulation on individual performance. F = Estimates population variance between each sample (between groups variance) Estimates population variance within each sample (within groups variance - individual differences) *Treatment variance = variation due to treatment (expected) *Error variance = variation due to random factors (i.e. - individual differences and experimental error) .F MSBT If the F-statistic is not significant, then there is no need to continue testing. However, a significant F-statistic indicates that a significant difference is present within the sample groups. However, the F-statistic does not indicate between which groups the significant differences occur. In Excel, the computed F-statistic is provided along with the critical value, in which our approach to making an inference about results is largely similar approach we used to conduct the hypothesis test by hand. In SPSS, we use the probability value, or p-value. The p-value refers to the probability of obtaining a F-statistic as extreme as the one observed between the samples if the hypothesis is true, in that the p-value represents the probability (%) of obtaining a test statistic more extreme than the test statistic computed. Therefore, p-values, in actuality, directly test whether our observed data agree or disagree with our null hypothesis itself. Whereas setting alpha to 0.05 in Excel seeks to limit the probability of making a Type I error between our observed versus expected differences as a function of our distribution; setting alpha to 0.05 relative to the p-value serves to set a threshold for how much we will allow our observed data to disagree with the null hypothesis. Therefore, if the p value is greater than .05, you will accept the null hypothesis (we went above threshold of accepted disagreement). If the p-value is less than or greater than .05, reject the null hypothesis (our results did not extend beyond threshold for disagreement with the null hypothesis). The problem with multiplicity occurs when you compare multiple groups, as there is an increased chance of Type I error for every pairwise comparison beyond 2 groups. To correct for this, the Bonferroni-Holm procedure may be used to calculate an adjusted alpha level for comparisons