Problem 4.3

Over the long term, a continuous-time Markov chain on {1, 2, 3, 4} spends 10% of the time in state 1, and 30% of the time each in state 2, 3, and 4, respectively. Moreover, from any state i {1, 2, 3, 4}, it stays on average i minutes before moving to a new state.

(a) Find the stationary distribution of the continuous-time chain.

(b) Find the stationary distribution of the embedded discrete-time chain.