A coin is then flipped and if it comes up heads, then the remaining states X1,... are obtained from the transition probability matrix P(1) and if tails from the matrix P(2)

. Is {Xn, n 0} a Markov chain?

If p = P{coin comes up heads}, what is limn→∞ P(Xn = i)?

(b) X0 = 1. At each stage the coin is flipped and if it comes up heads, then the next state is chosen according to P(1) and if tails comes up, then it is chosen according to P(2)

. In this case do the successive states constitute a Markov chain? If so, determine the transition probabilities. Show by a counterexample that the limiting probabilities are not the same as in part (a).