Joe wants to sell his car. He receives one offer each month and must decide immediately whether to accept the offer.

Once rejected, the offer is lost. The possible offers are $600, $800, and $1,000, made with probabilities 5

8

, 1 4

, and 1

8

, respectively (where successive offers are independent of each other). There is a maintenance cost of $60 per month for the car. Joe is anxious to sell the car and so has chosen a discount factor of 0.95.

Using the policy improvement algorithm, find a policy that minimizes the expected total discounted cost. (Hint: There are two actions: Accept or reject the offer. Let the state for month t be the offer in that month. Also include a state , where the process goes to state  whenever an offer is accepted and it remains there at a monthly cost of 0.)