At the beginning of each day, a piece of equipment is inspected to determine its working condition,

Question:

At the beginning of each day, a piece of equipment is inspected to determine its working condition, which is classified as state 1 = new, 2, 3, or 4 = broken. We assume the state is a Markov chain with the following transition matrix:

1 1 0.95 0 3 0 123 2 2 0.05 0.9 0 3 0 0.1 0.875 4 0 0 0.125

(a) Suppose that a broken machine requires 3 days to fix. To incorporate this into the Markov chain we add states 5 and 6 and suppose that p(4, 5) = 1, p(5, 6) = 1, and p(6, 1) = 1.

Find the fraction of time that the machine is working.

(b) Suppose now that we have the option of performing preventative maintenance when the machine is in state 3 and that this maintenance takes 1 day and returns the machine to state 1. This changes the transition probability to

1 1 0.95 72 2 0 3 1 2 0.05 0.9 0 3 0 0.1 0