Consider an irreducible continuous timeMarkov chain whose state space is the nonnegative integers, having instantaneous transition rates qi,j and stationary probabilities Pi , i ≥ 0. Let T be a given set of states, and let Xn be the state at the moment of the nth transition into a state in T .

(a) Argue that {Xn,n ≥ 1} is a Markov chain.

(b) At what rate does the continuous timeMarkov chain make transitions that go into state j .

(c) For i ∈ T , find the long run proportion of transitions of the Markov chain

{Xn,n ≥ 1} that are into state i.