Part 1: A/B Testing [40 points] Dataset: SteamAB.xlsx Temporary discounts get customers to buy games and engage with the platform. Recently, Steam decided it wanted to test whether price discounts offered to inactive users could get them to return to the platform and stick with it. They targeted two segments: Segment 1 Losing Interest: People whose activity playing Steam has dropped below three hours per week over the last two months. Segment 2 Inactive: People who have not played Steam for two to four months, but who have not uninstalled Steam. For each segment, 30,000 accounts were randomly selected. Two different treatment e-mails were sent to 10,000 accounts each (with the remaining 10K used as a control group): Incentive A 10% Discount: Congratulations! A 10% discount coupon has been credited to your account for use on any game! Incentive B 20% Discount: Congratulations! A 20% discount coupon has been credited to your account for use on any game! Over the next two months, purchase and game play data was collected for each of the accounts. Your boss has asked you to examine the data and report back on which incentive(s) should be used for each segment (if any) going forward, and why. Consider factors that may be diagnostic of potential customer lifetime value, in addition to short-term profitability.

In the past, Steam has also e-mailed existing customers to promote games from time to time. These e-mails are merely informativeno discount is attached. All customers received these e-mails; A/B tests were not run. As a consequence, we must identify the impact of these e-mails using a difference-in-differences approach. Youve been provided data for a two-week period. At the start of the second week, e-mail promotions were sent out for some games but not others (identified by the Promotion variable). Your task is as follows: (1) Run a basic difference-in-difference (using a treatment group dummy, treatment period dummy, and their interaction no other variables for now) to estimate the impact of the promotional e-mails. Report your regression results [10 points]. Interpret your interaction parameter estimate within the context of this case what does the difference-in-difference mean here? [10 points] Note that video games that receive the promotion are in the treatment group; other games are in the control group. Week 1 is the control period, week 2 is the treatment period. The data are not currently formatted to run a regression; you will need to re-organize the data to do so. Think carefully about how to re-arrange the data. (2) Once this is completed, examine the other variables in the dataset. How might you improve your regression by adding control variables? Explain why the variables you add should matter (examining the data by generating plots or summary statistics will help here) [10 points]. (3) Now run a regression with the variables you discussed in the previous question. Explore different functional forms for your control variables, and report your final specification [15 points]. (4) Your estimate for the difference-in-difference parameter (the causal effect of the promotion) will be slightly difference in your regression from question (3) than your regression from question (1), but it wont be much different. This lack of meaningful difference occurs in spite of the fact that many of your controls were strongly statistically significant, and your r-squared increased considerably. Why is that the case? [5 points] (5) Your regression results from part (1) should have shown a non-significant effect for the treatment group. In this context, this should not be surprising. Explain the interpretation of the treatment group parameter in this setting, and why it is not surprising that our estimate for it was zero. [3 points] (6) Our treatment group was all games that received a promotion, and our control group was all games that did not. Is there a better way to conduct our analyses in this setting? Explain your rationale, and run your proposed analyses to verify that it works better (i.e., that you learn something more than you did with the previous analyses). [7 points]