Question 1. A skincare company specializes in a new type of antioxidant vitamin C serum. The sales of this serum fluctuates depending upon two factors: (1) whether they advertise and (2) the advertising and marketing new products by competitors. The second factor is out of the company's control, but it is trying to determine what its own advertising policy should be. Each week there are two platform options to advertise: newspaper or TV. Placing an advertisement on newspaper costs $1000 per week, on TV costs $2000 per week. Weekly sales can be classified into three categories: high, average and low. Company believes that current week's sales depend probabilistically on both the previous week's sales and whether they advertise on newspaper or on TV. The probabilities are given in the table below. Previous Week's Sales High High 0.2 Average 0 Low 10 Newspaper This Week's Sales Average Low 0.5 0.3 0.6 0.4 0.3 0.7 High 0.6 0.4 0.2 TV This Week's Sales Average Low 0.3 0.1 0.5 0.1 0.7 0.1 The weekly profit (without taking advertising costs into account) from high sales is $15000, from average sales is $10000, and from low sales is $5000. Management now wants to determine the advertising policy that will maximize the company's (long-run) expected average net profit per quarter. Formulate the problem as an MDP and solve it via (a) Exhaustive enumeration, 1 (b) LP formulation, (c) Policy iteration. Question 1. A skincare company specializes in a new type of antioxidant vitamin C serum. The sales of this serum fluctuates depending upon two factors: (1) whether they advertise and (2) the advertising and marketing new products by competitors. The second factor is out of the company's control, but it is trying to determine what its own advertising policy should be. Each week there are two platform options to advertise: newspaper or TV. Placing an advertisement on newspaper costs $1000 per week, on TV costs $2000 per week. Weekly sales can be classified into three categories: high, average and low. Company believes that current week's sales depend probabilistically on both the previous week's sales and whether they advertise on newspaper or on TV. The probabilities are given in the table below. Previous Week's Sales High High 0.2 Average 0 Low 10 Newspaper This Week's Sales Average Low 0.5 0.3 0.6 0.4 0.3 0.7 High 0.6 0.4 0.2 TV This Week's Sales Average Low 0.3 0.1 0.5 0.1 0.7 0.1 The weekly profit (without taking advertising costs into account) from high sales is $15000, from average sales is $10000, and from low sales is $5000. Management now wants to determine the advertising policy that will maximize the company's (long-run) expected average net profit per quarter. Formulate the problem as an MDP and solve it via (a) Exhaustive enumeration, 1 (b) LP formulation, (c) Policy iteration
