MICHAEL LUCA HYUN'IN KIM Advertising Experiments at RestaurantGrades Shirley A. Critic, founder of RestaurantGrades (RG), a restaurant review platform (similar to Yelp or TripAdvisor) with an impressive stock Of Online reviews written by ordinary restaurant-goers, Was happy with RG's growth and position in the industry, and had compelling evidence that RG reviews had an important influence on the restaurant choices people made. However, doubts had surfaced about the efficacy Of the company's main source Of reeenueselling advertisements (ads) to restaurants. To better understand this issue, Critic decided to run a large-scale randomized controlled trial with a control group and two treatment groups: one to test the impact of RG's current ads on restaurant sales, and the other to test the impact of an alternative ad design to which she was considering switching. Her team ran the experiment and found the results in the spreadsheet attached in the data supplement. Being a data nerd (in the best possible sense) and knowing that the devil was Often in the details, Critic wanted to look at the results herself. She wanted to understand whether her company's ads really worked, and whether they should stick with the current design or switch to the alternative design. Background on RG Advertisements On RG, each restaurant had a profile page with operating information including its hours, phone number, and location, where RG users who visited the restaurant could leave reviews for other users. Users could also discover and search for restaurants on the platform using filters, and could make reservations and order food through a restaurant's profile page. The majority Of RG's revenues stemmed from selling ads through its sales team, which cold-called restaurants to try to convince them to advertise on the platform. Advertisements, labeled as sponsored search results, were placed in a separate section above the organic results for searches conducted by users. Packages of ads were purchased for about $300 per month, and advertisers were required to sign up for a one-year contract. While RG used a search algorithm much like Google' s that determined when and which ads were shown, depending on a user's search for restaurants on the platform, restaurants had little say in what search terms would trigger their ads. However, RG guaranteed that their ads would be shown a minimum of 1,000 times per month to users. s current search algorithm showed ads for restaurants triggered by type of cuisine within a 0.5- mile radius of a user's search. For example, if a user searched for Italian restaurants in Harvard Square, the algorithm chose two Italian restaurants within Harvard Square to advertise. The engineering team had run a variety of tests looking at how users responded to different types of ads in different searches, HWS Professor Michael Luca, Professor Weijia Dai (Lehigh University). and Doctoral Student Hyunjin Kim prepared this exercise as the basis for discussion rather than to illustrate either effective or ineffective handling of an administrative situation. Qlirley A Critic is fictional. Copyright and Fellows Of TO order copies or permission to reproduce materials, 545-7685, Write Harvard Business publishing, Boston, MA 02163, or go to This publication may not photocopied. or Otherwise reproduced, posted, or transmitted. Without the permission Of Harvard Business Sch'h'L This c:bcurnent is authorized Ior use only by Parsa Ghan ai in MGMT609-Fa112022 taught by Samuel Sarnbasivam. Wmdbury university from Oct 2022 to Jan 2023,