4 During any day, I may own either 0 or 1 share of a stock.

The price of the stock is governed by the Markov chain shown in Table 11. At the beginning of a day in which I own a share of stock, I may either sell it at today’s price or keep it. At the beginning of a day in which I don’t own a share of stock, I may either buy a share of stock at today’s price or not buy a share. My goal is to maximize my expected discounted profit over an infinite horizon (use b  .95).

a Use the policy iteration method to determine an optimal stationary policy.

b Use linear programming to determine an optimal stationary policy.

c Perform two iterations of value iteration.

d Find a policy that maximizes average daily profit.