Consider an ergodic continuous-time Markov chain, with transition rates 9.;, in steady state. Let P,, j 0, denote the stationary probabilities. Suppose the state space is partitioned into two subsets B and B = G.

(a) Compute the probability that the process is in state i, i B, given that it is in B That is, compute P{X(t)=iX(t) = B}.

(b) Compute the probability that the process is in state i, i B, given that it has just entered B That is, compute P{X(t) = i\X(t) = B, X(1) G}

(c) For i = B, let T, denote the time until the process enters G given that it is in state i, and let F,(s) = E[es] Argue that V F,(s) = [ F,(s) P., + P.,], jEB JEG P == where P,, = q.,,.,.

(d) Argue that 9 = IEGJEB

(e) Show by using

(c) and

(d) that = IER JEG SPF(s) P.q.,(1 - F,(s)) EB IEGJEB

(f) Given that the process has just entered B from G, let T, denote the time until it leaves B. Use part

(b) to conclude that E[e] = (g) Using

(e) and

(f) argue that ()P,9, JEB JEG ,4, JEG KEB , P.q., E[T]. JER LIEG JEB (h) Given that the process is in a state in B, let T, denote the time until it leaves B. Use (a), (e), (f), and (g) to show that E[es] 1- E[es] SE[T,] (i) Use (h) and the uniqueness of Laplace transforms to conclude that P{T, 1}= =So= P{T,>s} ds E[T] (j) Use (i) to show that E[T] = E[T] E[T] 2E[T] 2 The random variable T, is called the visit or sojourn time in the set of states B. It represents the time spent in B during a visit. The random variable T,, called the exit time from B, represents the remaining time in B given that the process is presently in B. The results of the above problem show that the distributions of T, and T, possess the same structural relation as the distributions of the excess or residual life of a renewal process at steady state and the distribution of the time between successive renewals.