5: Two-Way ANOVA Name: __________________________________ In this assignment, you will: 1. Enter raw data into an SPSS file 2. Conduct a Two-way ANOVA in SPSS 3. Compute effect sizes for any significant F-tests 4. With the help of SPSS plots, interpret the interaction using plain language A professor in physical education conducts an experiment to determine how intensity of physical exercise, and the time of day the exercise is performed, affects the number of hours a person sleeps. The professor hypothesizes that the effect of exercise intensity on sleep will depend on the time of day in which the exercise is done. There are three levels of exercise intensity (light, moderate, heavy) and two times of day (morning, evening). Thirty-six college students who are in good physical condition are randomly assigned to the six cells such that there are six subjects per cell. The subjects who do heavy exercise jog for three miles, the subjects who do moderate exercise jog for one mile, and the subjects in the light exercise condition jog for only mile. Morning exercise is done at 7:30 a.m., whereas evening exercise is done at 7:00 p.m. Each subject exercises once and the number of hours slept that night is recorded. The gathered data are given below. LENGTH OF EXERCISE Light Morning Moderate Heavy 6.5 7.3 6.6 7.4 7.2 6.8 6.1 7.7 7.1 7.6 6.6 7.4 7.3 6.8 6.7 7.3 7.5 7.3 7.2 7.9 8.2 7.7 7.5 7.7 7.4 8.1 8.2 8.0 7.6 8.4 8.2 8.5 9.5 8.7 9.6 9.5 TIME OF DAY Evening 1. To set up your Two-way ANOVA: a. Go to \"Analyze\` Assignment 6: Interactions and Post-Hoc Tests in Two-Way ANOVA Name: __________________________________ In this assignment, you will: 1. Examine post-hoc tests of the two-way ANOVA, computing Q-statistics and effect sizes. 2. Decompose and make sense of the interaction, reporting simple main effects and effect sizes. 3. Report the results of your entire two-way ANOVA analysis using APA format. This week we continue with our analysis of the following dataset, which you have already entered into SPSS and used to produce a two-way ANOVA output file. 1. You will need to re-run your analysis to get SPSS to conduct the post-hoc tests and break down the interaction. To set up your full Two-way ANOVA analysis: a. Go to \"Analyze\Chapter 16: Interactions and Post-Hoc Analyses for Two-Way ANOVA 1 EDPR 7/8542 Announcements/Questions Midterm on Oct 12th Next week: review session Questions about Q-statistics and Cohen's d: What does it mean if they're negative? When interpreting the size of Cohen's d, ignore the negative sign (i.e., just look at absolute value) EDPR 7/8542 Review of Two-Way ANOVA What are three advantages of conducting two-way ANOVAS? (vs. separate one-way ANOVAs) 1) Error reduction due to addition of another IV 2) Research efficiency: same dataset and participants can be used to answer multiple research questions 3) Ability to examine presence of interaction effects The two-way ANOVA output will show you whether each main effect is significant What's a \"main effect\"? i.e., whether the IV has a significant effect on the DV If the main effect is significant, what does one do next? Conduct post-hoc tests The two-way ANOVA output will show you whether the interaction is significant. What's an \"interaction\"? If interaction is significant, you need to decompose it i.e., do \"simple main effect\" analyses EDPR 7/8542 Review of Two-Way ANOVA EDPR 7/8542 Review of Post-Hoc Tests Procedure for each post-hoc: SPSS will compute p-values Compute Q-statistic by hand Compute effect size by hand (Cohen's d is easiest) Review: which post-hoc test do I use? If have equal variances and equal group sizes: Report Tukey HSD Remember: n in formula represents number of scores in one level of the IV If have equal variances but unequal group sizes: Report Tukey-Kramer Compute harmonic n by hand Or check SPSS output under Homogeneous Subsets table Unequal variances (regardless of group sizes): Report p-value for Games-Howell EDPR 7/8542 But what statistic??? Conducting Follow-Up Tests Results of post-hoc analysis Notice: Analysis is collapsed across the age groups What do the results show? All three dosages differ significantly from each other EDPR 7/8542 Conducting Follow-Up Tests To compute Q, use your Descriptives table to figure out the n For the 100mg and 50mg groups: n = 12 What's MSerror? Find in ANOVA table: MSerror = .539 What's the effect size? EDPR 7/8542 Breaking Down Interactions EDPR 7/8542 What does it mean if the interaction is significant? The effect of drug dosage depends on age But what dosage at what age? We can plot the interaction to get a better understanding of it The plots help, but we still don't know which pairs of cells are statistically different 8 Breaking Down Interactions EDPR 7/8542 Thus, if we have a significant interaction, we must conduct tests of simple main effects (see p. 419) We will not compute these by hand Procedure (conceptually): You want to make sense of the age*dosage interaction 1) Do one-way ANOVA (and post-hocs) for just the high school participants Find out which dosages differ from each other in high school students 2) Do a one-way ANOVA (and post-hocs) for just the college participants Find out which dosages differ from each other in college students 3) Compare the two ANOVAs to get a complete understanding of the interaction 9 /EMMEANS=TABLES(AGE*DOSAGE) COMPARE(AGE) ADJ(Bonferroni) Breaking Down Interactions in SPSS SPSS doesn't by default conduct tests of simple main effects By default, only reports p-value of the interaction We will make changes to SPSS syntax file to get these analyses Note: when using syntax commands, only Bonferroni and Sidak corrections are available In SPSS syntax window, add: /EMMEANS=TABLES(AGE*DOSAGE) COMPARE(DOSAGE) ADJ(Bonferroni) This will break down the interaction by age and make a Bonferroni adjustment Also, directly below that add: This will break down the interaction by dosage and make a Bonferroni adjustment One of these will be easier to understand, focus on it when reporting your results (and ignore the other) EDPR 7/8542 Decomposing Interactions in SPSS 11 EDPR 7/8542 By Age 12 EDPR 7/8542 By Dosage For me, this table does a more intuitive job of communicating the results (the choice is personal) I would focus on this table when reporting my results, and ignore the previous table EDPR 7/8542 By Dosage Standard Error found in table above df = n - 2 (number of people in both groups - 2) EDPR 7/8542 When conducting the Bonferroni post-hoc, SPSS gives the p-value, but not the t-statistic nor the Cohen's d Thus, must compute t-statistic and Cohen's d by hand 1. 2. 3. 4. 5. 6. Reporting Results in APA Format Describe analysis that was conducted \"A two-way ANOVA was performed to determine if...\" Discuss whether assumptions were met Discuss outcome of 1st F-test (main effect for first IV) And subsequent post-hocs, if necessary Include Q-statistic and effect sizes Discuss outcome of 2nd F-test (main effect for second IV) And subsequent post-hocs, if necessary Include Q-statistic and effect sizes Discuss outcome of interaction F-test (interaction effect) Include effect size Discuss the details of the interaction Break down the interaction and report results of simple main effects i.e., include t-statistics and effect sizes Adding a figure here is helpful! EDPR 7/8542 Final Comments 1) Whether an interaction is present is unrelated to presence of main effects. Any of the following can occur: Interaction and both main effects present Interaction and no main effects present Interaction and only one main effect present 2) Some researchers believe it is unnecessary to report main effects if the interaction is significant i.e., if interaction is significant, don't report any statistics about the main effects This is because the interaction presents the most nuanced/detailed description of the results I disagree: Main effects may be worthwhile to report, depending on specific research questions of the study Thus, in this course, we will always report significant interactions AND main effects EDPR 7/8542 Final Comments 3) The two-way ANOVA in this example has two betweensubjects variables. In real-life research, a two way ANOVA often has one within-subjects variable and between-subjects variable And intervention type (new therapy, usual therapy, control group) E.g., intervention duration (e.g., at start, after 1 month, after 2 months)... This is called a mixed between-within ANOVA What would you look for in the output to see if therapy is effective? The interaction between intervention and duration because it tells you if the therapy group(s) changed differently from the control group over time EDPR 7/8542 Results A two-way ANOVA was conducted to determine if dosage of anti-anxiety medication (0mg, 50mg, 100mg) and participant age (high school age, college age) affect algebra test performance. The Levene's test was not significant, F(5,30) = .76, p = .587; thus, the homogeneity of variance assumption was met. 2 The main effect for age was on the verge of statistical significance, F(1,30) = 4.18, p = .050, = .01, suggesting that high-school-age participants (M = 5.17, SD = 1.76) score significantly higher than college-age participants (M = 4.67, SD = 3.03). A significant effect was also present for dosage, F(2,30) = 162.53, p < .001, 2 = .82. Post-hoc Tukey tests indicated that all three pairwise comparisons were statistically significant: the 50mg group (M = 4.42, SD = .79) scored significantly higher than the 0mg group (M = 2.50, SD = 1.38), Q(k = 3, df = 30) = 9.06, p < .001, d = 2.62; the 100 mg group (M = 7.83, SD = .84) scored significantly higher than the 50mg group, Q(k = 3, df = 30) = 16.14, p < .001, d = 4.66; and the 100mg group scored higher than the 0mg group, Q(k = 3, df = 30) = 25.15, p < .001, d = 7.26. There was also a significant interaction between dosage and age, F(2,30) = 15.93, p < .001, 2 = .08 (see Figure 1). Follow-up tests revealed that at the 0mg dosage, high-school-age participants (M = 3.67, SD = .82) scored significantly higher than college-age participants (M = 1.33, SD = .52), t(10) = 5.50, p < .001, d = 3.18. At 50mg, there was no difference between high-school-age participants (M = 4.50, SD = .84) and college-age participants (M = 4.33, SD = .82). Lastly, at 100mg, college-age participants (M = 8.33, SD = .82) scored significantly higher than high-school-age participants (M = 7.33, SD = .52), t(10) = 2.36, p = .025, d = 1.36. Overall, the interaction suggests that changes in drug dosage have a stronger effect on college students than on high school students. Figure 1: The Interaction between Dosage and Age on Algebra Test Performance Comment [MMM(1]: Note: the degrees of freedom represent the degrees of freedom associated with MS error, which in this case is 30