Mini Job (MJ) is a small job shop manufacturer with a contract to stamp 200 copies per day for the next 5 days of a metal door panel needed by an automobile manufacturer. MJ’s machining center that does the stamping can meet that demand with regular 40-hour shifts, costing a total of $3000 in labor per day if the machine is full

“working” state i = 1 in which it begins. But if the device has “minor defects” (state i = 2), productivity falls to 160 copies within regular hours, leaving the remaining 40 to be done on overtime at a total of $4500 per day. If the the device has

“major defects” (state i = 3), only 75 units can be produced in regular hours, 25 more on the overtime shift, and the other 100 must be purchased from a competitor at a total of $10,000 per day.

Finally, the machine may be completely “inoperable”

(state i = 4). On such days NJ must contract the full 200 units at a total cost of $20,000.

MJ’s planning challenge is to decide when to do repair on the work center. On any day, MJ can leave the machine in its current condition, or undertake repairs. In full working state i = 1, the machine will continue in that state with probability 0.60 and decline to state i = 2 with probability 0.40. Without repair of minor defects, the machine may continue in state i = 2 for another day at probability 0.50, or decline to state i = 3 or i = 4 with probabilities 0.30 and 0.20, respectively. Similarly, without repair of major defects the machine will continue state i = 3 with probability 0.40 and become inoperable with probability 0.60. Assume repair happens instantly once a decision is made.

If repairs are made to the machine in minor defects state i = 2 at cost $9000, there is a 0.90 chance it will return to full working state, and 0.10 that it continues to have minor defects. Similarly, if repairs are made to the machine in major defects state i = 3 at cost $14,000, there is a 0.70 chance of restoring full working state, 0.20 of moving up to minor defects state i = 2, and 0.10 of continuing to have major defects. Doing repairs on the center when it is completely inoperable, would cost $17,000, but have a 0.60 chance of making it fully operable, a 0.20 chance of leaving it with minor defects, a 0.15 chance of moving it only to major defects state, and 0.05 of leaving the machine still inoperable.

MJ seeks a policy for when to repair that minimizes the total expected cost of its operations over the 5-day contract period.

(a) Formulate MJ’s challenge as a MDP with multiple states and stages, including identifying all the elements of definition 9.44 .

(b) Sketch a digraph depicting your model of part

(a) over nodes for the states and stages, plus arcs for possible transitions.

You need not insert all parameter details, but do show exemplars of all transition arcs with the decision to which they are attached, the reward they would realize, and the associated probability.

(c) Form the functional equations for all states and stages of your model in (a).

(d) Solve the equations of

(c) to determine an optimal repair policy for MJ.