Q#5. Markov chains You may use computational software for calculations, but express your answers using proper mathematical notation. Let x, t {1,2,...), be a homogenous Markov chain taking states X {1,2,...,5} with one-step transitional probability matrix and initial distribution P = respectively. (a) Calculate var(x,). 1/10 3/10 0 0 5 5 5 P(0) = 6 3 6 6 1/8 1/8 1/2 01/31/01/2 (b) Calculate a stationary distribution T. 2/5 0 0 0 0 0 -1 3/8 1/8 4 0 Let Y, t 0, be a homogenous Markov chain taking states Y {1,2,...,5} with generator matrix (d) Let Y= = 3 and let the waiting time What is E[T3]? 1 A = 1 1/2 -31/5/ 4 1 -1 % 4 6 6 0 0 0 0 0 (c) What is the probability of the state change 3-4 when the Markov chain jumps? T = min(s>0|Y4+53). 13/10 (e) A Markov chain is ergodic if the limit of the state probability vector, lim p(t), exists and does not depend on p(t). Using this criterion, determine if Y, ergodic? t-00 Q#5. Markov chains You may use computational software for calculations, but express your answers using proper mathematical notation. Let X, t = {1,2,...), be a homogenous Markov chain taking states X {1,2,...,5} with one-step transitional probability matrix and initial distribution P = respectively. (a) Calculate var(x,). 1/10 3/10 0 0 5 5 5 P(0) = 6 3 6 6 1/8 1/8 1/2 01/31/01/2 (b) Calculate a stationary distribution T. 2/5 0 0 0 0 0 -1 3/8 1/8 4 0 Let Y, t0, be a homogenous Markov chain taking states Y {1,2,...,5} with generator matrix (d) Let Y= = 3 and let the waiting time What is E[T3]? 1 A = 1 1/2 -31/5/ 4 1 -1 % 4 6 6 0 0 0 0 0 (c) What is the probability of the state change 3-4 when the Markov chain jumps? T = min(s>0|Y4+53). 13/10 (e) A Markov chain is ergodic if the limit of the state probability vector, lim p(t), exists and does not depend on p(t). Using this criterion, determine if Y, ergodic? t-00