1. All else equal, which of the following consequences does increasing the sample size in an experiment have? (Select all that apply)
   1. Reduces the width of the confidence interval for the estimated treatment effect
   2. Increases the size of the estimated effect of the experimental treatment
   3. Reduces the chances of a false positive result
   4. Increases the statistical power
   5. Makes it more likely that the result will be statistically significant if the treatment is effective in reality
   6. Increases the internal validity
2. An experiment was conducted on Twitter to compare the effectiveness of celebrity endorsement ads versus product comparison ads in driving traffic to a retail site. Identify each of the problems below as an issue of failed randomization, statistical power, reliability, internal validity or external validity, and explain why. (There is some overlap between these concepts, so you can list more than one if you like.)
   1. The celebrity in the ad was arrested for a DUI right before the experiment started running.
   2. Instead of running for a week as planned, the ads only ran for two days.
   3. Due to a computer glitch, site visitors coming from Bing (who had not been exposed to either ad) were mistakenly coded as coming from the product comparison ad.
   4. At some unidentified point during the last day of the experiment, an edit to the software caused visitors from the celebrity ad to be mistakenly coded as coming from the product comparison ad, and vice versa.
   5. Fans of the celebrity re-tweeted the ad, and the data doesn't distinguish between visitors who were exposed to the ad campaign or the retweeted ad.

c) Which of the following would typically increase statistical power in an experiment testing between two treatments (assuming no unintended consequences)?

1. Increasing the total sample size
2. Planning your analysis in advance
3. Taking more measurements of the outcome of interest and averaging them
4. Keeping the sample size the same but reducing the number of experimental conditions
5. Including all participants whether they received the treatment or not
6. Testing the effect for a subgroup of participants
7. Testing the effect in a regression, controlling for behavior prior to the experiment