Consider a system that alternates between the two states 0 (OFF) and 1 (ON) and that is checked at discrete time points. If the system is OFF at one time point, the probability that it has switched to ON at the next time point is , and if it is ON, the probability that it switches to OFF is .

(a) Since the Markov property is satisfied, the system can be modeled by a Markov chain. Draw the transition graph and write down the transition matrix of the Markov chain.

(b) Suppose that = 3/4 and = 1/2 . If the system starts being OFF at time = 0, what is the probability that it is ON at time = 3?

(c) Find the stationary distribution for the Markov chain in terms of and . What do you conclude about the proportion that the system stays at OFF and ON in a long time period?