Let (Xn)n0 be a Markov chain with state space f0; 1; 2; : : :g. Let (i)i0 be the initial

distribution of the chain, and (Pij)i;j0 be its transition probabilities.

(a) (10 marks) Calculate i = E(X1jX0 = i) as a function of (Pij)i;j0. Then, calculate the

covariance = Cov(X0;X1) as a function of (i)i0 and (i)i0.

(b) (10 marks) Let a 2 (0; 1) and b 2 (0; 1) be xed. For any i 0, let pi = bi. Assume that:

i = (1 ab)aipi for any i 0

Pij = (1 pi)j1pi; for all j 1; and Pi0 = 0:

Use the result in part (a) to calculate = Cov(X0;X1) as a function of a and b.

Hint: If X is a geometric random variable with P(X = n) = (1 p)n1p for n = 1; 2; 3; : : :

and p 2 (0; 1), then E(X) = 1=p.