Answer all questions

(p) All Markov chains must have a finite number of states. (q) All irreducible Markov chains must have a finite number of states. (r) All irreducible Markov chains are periodic. (s) All irreducible Markov chains are aperiodic. (t) All discrete-time Markov chains are irreducible.Let P be the transition matrix of a Markov chain with 7 states. Which one of the following statements is not always true? O p2 is the transition matrix of a Markov chain with 72 states. If O is another transition matrix of a Markov chain with 72 states, then PQ is the transition matrix of a Markov chain with 72 states. O If ) is another transition matrix of a Markov chain with 72 states, then *(P + Q) is the transition matrix of a Markov chain with 7, states. If P is invertible, then p-1 is the transition matrix of a Markov chain with 71, states.Consider a Markov chain {Xn, n = 0, 1, . ..} on the state space S = {0, 1, 2}. Suppose that the Markov chain has the transition matrix 2 10 10 10 2 P = 3 10 2 4 10 10 1. Show that the Markov chain has a unique stationary mass. 2. Let h denote the stationary mass of the Markov chain. Find h(x) for all x E S. 3. Show that the Markov chain has the steady state mass. 4. Let h* denote the steady state mass of the Markov chain. Find h*(x) for all x E S