Applied Probability

Let (Xn)no be a Markov chain with state space {0, 1, 2, ...}. Let (ai)izo be the initial distribution of the chain, and (Pij),j>0 be its transition probabilities. (a) (10 marks) Calculate Mi = E(X1|Xo = 2) as a function of (Pij)i,j>0. Then, calculate the covariance p = Cov(Xo, X1) as a function of (ai)izo and (Mi)izo. (b) (10 marks) Let a E (0, 1) and b E (0, 1) be fixed. For any i 2 0, let pi = b'. Assume that: ai = (1 - ab)alpi for any 2 2 0 Pij = (1 -pi) pi, for all j 2 1, and Pio = 0. Use the result in part (a) to calculate p = Cov(Xo, X1) as a function of a and b. Hint: If X is a geometric random variable with P(X = n) = (1 -p)"-'p for n = 1, 2, 3, ... and p E (0, 1), then E(X) = 1/p