Markov chain

A stationary distribution of an m-state Markov chain is a probability vector q such that = q P, where P is the probability transition matrix. A Markov chain can have more than one stationary distribution. Identify all the stationary distributions that you can, for the 3-state Markov chain with transition probability matrix O O P Owl Does this Markov chain have a steady-state probability distribution ? 15 points2. A Markov chain with state space {1, 2, 3} has transition probability matrix 0.6 0.3 0.1 [Pa = 0.3 0.3 0.4 0.4 0.1 0.5 (a) Is this Markov chain irreducible? Is the Markov chain recurrent or transient? Explain your answers. (1)] What is the period of state 1? Hence deduce the period of the remaining states. Does this Markov chain have a limiting distribution? (0) Consider a general three-state Markov chain with transition matrix P11 P12 P13 1?: 1021 1022 p23 P31 P32 P33 Give an example of a specic set of probabilities pm; for which the Markov chain is not irreducible (there is no single right answer to this, of course l]. Let {Xiji=1 36 be the sequence of continuous IIDRV with EXi = 40, Var Xi = 144, i = 1, ..., 36 and A36 Eil Xi = 36 (a) By using the Markov's inequality estimate P( A36