Let {X,, n 1} denote an irreducible Markov chain having a countable state space. Now consider a new stochastic process {Y,, n 0} that only accepts values of the Markov chain that are between 0 and N That is, we define Y, to be the nth value of the Markov chain that is between O and N For instance, if N = 3 and X = 1, X = 3, X = 5, X = 6, X, 2, then Y = 1, Y = 3, Y = 2. =

(a) Is {Y, n0} a Markov chain? Explain briefly

(b) Let 77, denote the proportion of time that {X, n = 1} is in state j If , > 0 for all j, what proportion of time is {Y,, n = 0} in each of the states 0, 1, ..., N?

(c) Suppose {X} is null recurrent and let ,(N), i = 0, 1, the long-run proportions for {Y,, n 0} Show that ,(N),(N)E [time the X process spends in j , N denote between returns to i], ji.

(d) Use

(c) to argue that in a symmetric random walk the expected number of visits to state i before returning to the origin equals 1

(e) If {X, n0} is time reversible, show that {Y, n 0} is also