We have a machine that is in one of five possible conditions at each time. State 1 is the best condition, etc. down to state 5 , which is the worst condition. We can replace a machine with one that is in condition 1 at any time. If we do not replace, the machine in operation will stay in its current condition with probability \(3 / 4\), or go to the next worst condition with probability \(1 / 4\) at the next instant of time. If the machine is currently in the worst condition 5, it will stay there until it is replaced. There are costs for inferior production in each period, dependent on the machine's condition:

A new machine costs a constant \(C_{6}=12\) monetary units. What replacement strategy should be adopted in order to minimize expected total discounted cost, with a discount factor \(\alpha=9 / 10\) ? For your solution, use policy improvement with the initial policy of replacing the machine whatever is its condition.

Transcribed Image Text:

C =0, C2, C3 = 4, C4 = 6, C5 = 8.