Finding a P-value by simulation. Is a new method of teaching reading to first-graders (Method B) more effective than the method now in use

(Method A)? You design a matched pairs experiment to answer this question.

You form 20 pairs of first-graders, with the two children in each pair carefully matched in IQ, socioeconomic status, and reading-readiness score. You assign at random one student from each pair to Method A. The other student in the pair is taught by Method B. At the end of first grade, all the children take a test of reading skill. Let p stand for the proportion of all possible matched pairs of children for which the child taught by Method B has the higher score. Your hypotheses are H0: p = 0.5 (no difference in effectiveness)

Ha: p > 0.5 (Method B is more effective)

The result of your experiment is that Method B gave the higher score in 12 of the 20 pairs, or ^p = 12/20 = 0.6.

(a) If H0 is true, the 20 pairs of students are 20 independent trials with probability 0.5 that Method B wins each trial. Explain how to use Table A to simulate these 20 trials if we assume for the sake of argument that H0 is true.

500 CHAPTER 22 What Is a Test of Significance?

(b) Use Table A, starting at line 105, to simulate 10 repetitions of the experiment.

Estimate from your simulation the probability that Method B will do better in 12 or more of the 20 pairs when H0 is true. (Of course, 10 repetitions are not enough to estimate the probability reliably. Once you see the idea, more repetitions are easy.)

(c) Explain why the probability you simulated in

(b) is the P-value for your experiment.

With enough patience, you could find all the P-values in this chapter by doing simulations similar to this one.