While developing their service plan, FroCorp would also like to model how
often they need to service an entire store based on keeping a minimum of
2 machines operational (they have a total of 4 machines). To do this the
project manager has asked you to construct a Markov chain for the number
of working machines at the start of each business day. You may assume that
the probability of a working machine breaking during each business day is
p independent of all other machines and previous days. A service will occur
outside of business hours if less than 2 machines are operational at the end
of a business day, ensuring that all 4 machines are working at the start of
the next day. You have been tasked with finding the following:
(a) Determine the transition probability matrix of this Markov chain.
(b) If 2 machines are working on Wednesday morning, what is the probability
that 2 machines are working on Friday morning (you do not need to
simplify the expression).

(c) Determine the classes of this Markov chain and their type, including the
periodicity of any recurrent classes.
(d) Does this Markov chain have a unique stationary distribution? Explain
your answer.
(e) Is the limiting and stationary distributions the same for this Markov
chain? Explain your answer.