Answer appropriately

Consider a Markov chain {Xn, n = 0, 1, . ..} on the state space S = {0, 1, 2}. Suppose that the Markov chain has the transition matrix 2 10 10 10 2 P = 3 10 2 4 10 10 1. Show that the Markov chain has a unique stationary mass. 2. Let h denote the stationary mass of the Markov chain. Find h(x) for all x E S. 3. Show that the Markov chain has the steady state mass. 4. Let h* denote the steady state mass of the Markov chain. Find h*(x) for all x E S.2. A Markov chain with state space {1, 2, 3} has transition probability matrix 00 0.3 0.1 a: 0.3 0.3 0.4 0.4 0.1 0.5 (a) Is this Markov chain irreducible? Is the Markov chain recurrent or transient? Explain your answers. (b) What is the period of state 1? Hence deduce the period of the remaining states. Does this Markov chain have a limiting distribution? (c) Consider a general three-state Markov chain with transition matrix 3011 3012 1013 P = P21 P22 P23 1031 P32 P33 Give an example of a specic set of probabilities jag-'3; for which the Markov chain is not irreducible (there is no single right answer to this1 of course l]. 4. Consider a discrete-time Markov chain with the following probability transition matrix 0 0 0 0 7 1-3-y P = 1-I-VVO T 0 0 1 0 Is it possible to choose values for a and y so that the Markov chain has the following properties? In each case, state the values of a and y, or give a brief reason why it is not possible. (a) The Markov chain has period 2. [2) (b) The Markov chain is reducible. (c) The Markov chain has at least one transient state. UNN (d) The Markov chain has invariant distribution (1/4, 1/4, 1/4, 1/4).1. Consider the Markov chain with the following transition matrix. 0 0.5 0.5 0.5 0 0.5 0.5 0.5 0 (a) Draw the transition diagram of the Markov chain. (b) Is the Markov chain ergodic? Give a reason for your answer. (c) Compute the two step transition matrix of the Markov chain. (d) What is the state distribution *2 for t = 2 if the initial state distribution for t = 0 is no = (0.1, 0.5, 0.4)