Let us expand the model of Example 2.

Suppose now that the machine can be in "like new" condition (state 0), "badly deteriorated" condition (state 4), or one of three intermediate states of deterioration, labeled \(1,2,3\), in order of increasing severity. Again suppose that we do not act if the machine is in state 0 , we must replace the machine if it is in state 4 , and for the intermediate states we have the option of doing nothing, attempting a repair, or replacing the machine. Suppose that if we do nothing, the machine stays in its current state with probability \(3 / 4\) and reduces to the next lower state with probability \(1 / 4\) (except that if it is badly deteriorated, it stays in that state). If the repair option is chosen, the machine goes to the next higher level with probability \(1 / 2\), or stays the same with probability \(1 / 2\). Replacement always restores the state to "like new." Suppose that the costs of poor output for the five states are \(0,3,6,9\), and 12 ; the repair cost is 4 ; and the replacement cost is 10 . Let the discount factor \(\alpha\) be .95 . Find the optimal policy.