Q2(2) and Q3(2)

Q. 2 (12 points). Let (Xn) be a 3-state Markov chain with transition matrix . 1/2 1/4 1/4 P =2 1/4 1/2 1/4 3 0 3/4 1/4 The top row corresponds to state 1, the middle to state 2, and the bottom to state 3. (1) (6 points) Compute the expected value of the first time that the chain reaches the state 3, given that the chain starts from state 1. (2) (6 points) Compute the expected value of the oth time that the chain reaches the state 3, given that the chain starts from state 1. Q. 3 (40 points). Consider the time-homogeneous Markov chain on {0, . . . . N}. where N 2 1, which we will call the "slow walk around the the circle." At each time it either stays put, with probability q, or moves up by 1, with probability p. Here, p + q = 1 and 0