Exercise 4.62 describes an experiment involving playing games in class. One concern in the experiment is that the exam question related to Game 1 might be a lot easier or harder than the question for Game 2. In fact, when they compared the mean performance of all students on Question 1 to Question 2 (using a two-tailed test for a difference in means), they report a p-value equal to 0.0012.

(a) If you were to repeat this experiment 1000 times, and there really is no difference in the difficulty of the questions, how often would you expect the means to be as different as observed in the actual study?

(b) Do you think this p-value indicates that there is a difference in the average difficulty of the two questions? Why or why not?

(c) Based on the information given, can you tell which (if either) of the two questions is easier?  

Exercise 4.62

Two professors at the University of Arizona were interested in whether having students actually play a game would help them analyze theoretical properties of the game. The professors performed an experiment in which students played one of two games before coming to a class where both games were discussed. Students were randomly assigned to which of the two games they played, which we’ll call Game 1 and Game 2. On a later exam, students were asked to solve problems involving both games, with Question 1 referring to Game 1 and Question 2 referring to Game 2. When comparing the performance of the two groups on the exam question related to Game 1, they suspected that the mean for students who had played Game 1 (μ₁) would be higher than the mean for the other students μ₂, so they considered the hypotheses H₀: μ₁ = μ₂ vs H_a: μ₁ > μ₂.