Let T = {1,..., t} denote the transient states of a Markov chain, and let be, as in Section 44, the matrix of transition probabilities from states in 7 to states in T. Let m,,(n) denote the expected amount of time spent in state j during the first n transitions given that the chain begins in state i, for i and j in T. Let M, be the matrix whose element in row i, column j, is m,,(n)

(a) Show that M =I+Q+Q2 + ... + Q".

(b) Show that M

(c) Show that M, - = I + Q = Q1 + Q + Q + - - (IQ) (IQ"+1) +Q"1.