Exercise 7.2 A Markov chain (Xn)n=0,1,... with state space S = {1, 2, 3, 4, 5, 6, 7} has the following transition graph: 1/2 n @ 1/2 0.1 (a) Determine all communication classes. Is the Markov chain irreducible? (b) Determine all recurrent and all transient states. Exercise 7.3 Let (Xn)n=0717_,_ be a Markov chain with state space S and transition probability matrix P. Suppose that the Markov chain has two stationary distributions 7r1 and 7T2. Show that for any constant c 6 [0,1] the convex combination of 771 and 7T2 given by 7rc=c-7r1+(1c)~7r2 is again a stationary distribution