Exercise 4.153 explores the question of whether mate choice improves offspring fitness in fruit flies, and describes two seemingly identical experiments yielding conflicting results (one significant, one insignificant). In fact, the second source was actually a series of three different experiments, and each full experiment was comprised of 50 different mini-experiments (runs), 10 each on five different days.

(a) Suppose each of the 50 mini-experiments from the first study were analyzed individually. If mating choice has no impact on offspring fitness, about how many of these \(50 \mathrm{p}\)-values would you expect to yield significant results at \(\alpha=0.05\) ?

(b) The 50 p-values, testing the alternative \(H_{a}\) : \(p_{C}>p_{N C}\) (proportion of flies surviving is higher in the mate choice group) are given below:

\begin{array}{lllllllllll}
Day 1: & 0.96 & 0.85 & 0.14 & 0.54 & 0.76 & 0.98 & 0.33 & 0.84 & 0.21 & 0.89 \\
Day 2: & 0.89 & 0.66 & 0.67 & 0.88 & 1.00 & 0.01 & 1.00 & 0.77 & 0.95 & 0.27 \\
Day 3: & 0.58 & 0.11 & 0.02 & 0.00 & 0.62 & 0.01 & 0.79 & 0.08 & 0.96 & 0.00 \\
Day 4: & 0.89 & 0.13 & 0.34 & 0.18 & 0.11 & 0.66 & 0.01 & 0.31 & 0.69 & 0.19 \\
Day 5: & 0.42 & 0.06 & 0.31 & 0.24 & 0.24 & 0.16 & 0.17 & 0.03 & 0.02 & 0.11
\end{array}

How many are actually significant using \(\alpha=0.05\) ?

(c) You may notice that two p-values (the fourth and last run on day 3) are 0.00 when rounded to two decimal places. The second of these is actually 0.0001 if we report more decimal places. This is very significant! Would it be appropriate and/or ethical to just report this one run, yielding highly statistically significant evidence that mate choice improves offspring fitness? Explain.

(d) You may also notice that two of the p-values on day 2 are 1 (rounded to two decimal places). If we had been testing the opposite alternative, \(H_{a}\) : \(p_{C}

(e) Using the problem of multiple testing, explain why it is possible to get such contradictory results.

(f) In fact, the replication studies involved three different sets of experiments, each with a different type of mutant fly introduced for competition, and with two different species of fruit flies being studied. While the original Nature study yielded statistically significant results in favor of mate choice, and the first replication study yielded insignificant results, the latter two replication studies yielded statistically significant results in favor of no mate choice. Explain why replication of studies (the replication of a study in a different setting or by a different researcher) is an important part of science.

Data From Exercise 4.154:

Does the ability to choose a mate improve offspring fitness in fruit flies? Researchers have studied this by taking female fruit flies and randomly dividing them into two groups; one group is put into a cage with a large number of males and able to freely choose who to mate with, while flies in the other group are each put into individual vials, each with only one male, giving no choice in who to mate with. Females are then putinto egglaying chambers, and a certain number of larvae collected. Do the larvae from the mate choice group exhibit higher survival rates? A study published in Nature found that mate choice does increase offspring fitness in fruit flies (with p-value < 0.02), yet this result went against conventional wisdom in genetics and was quite controversial. Researchers attempted to replicate this result with a series of related experiments, with data provided in MateChoice.