Answers needed

5. A Markov chain {X,, n 2 0} with state space { 1, 2, 3, 4, 5 } has the transition probability matrix given below. DO P = Ooouo ooouo OHOOO HOO OOH (a) Draw the transition diagram for this Markov chain. (b) Is this Markov chain irreducible? Explain. (c) Determine the period of each state in this Markov chain.For each of the following transition matrices, determine whether the Markov chain with that transition matrix is regular: (1) Is the Markov chain whose transition matrix whose transition matrix is 1 0.2 0.8 0 0 regular? (Yes or No) (2) Is the Markov chain whose transition matrix whose transition matrix is 0 1 0.1 0.4 0.5 0 1 regular? (Yes or No) (3) Is the Markov chain whose transition matrix whose transition matrix is 0 0 0.8 0 0.2 0.8 0.2 0 regular? (Yes or No) (4) Is the Markov chain whose transition matrix whose transition matrix is 0 1 0.8 0 0.2 0 1 0 regular? (Yes or No)1. Consider the Markov chain with the following transition matrix. 0 0.5 0.5 0.5 0 0.5 0.5 0.5 0 (a) Draw the transition diagram of the Markov chain. (b) Is the Markov chain ergodic? Give a reason for your answer. (c) Compute the two step transition matrix of the Markov chain. (d) What is the state distribution *2 for t = 2 if the initial state distribution for t = 0 is no = (0.1, 0.5, 0.4) ?Problem 7.4 (10 points) A Markov chain X0,X1, X2, . . . with state space S = {1, 2, 3, 4} has the following transition graph: (a) Provide the transitiOn matrix fer the Markov chain. (13) Determine all recurrent and all transient states