Let (X_n)n0 be a Markov chain with state space {0, 1, 2, . . .}. Let (_i)i0 be the initial distribution of the chain, and (P_ij )i,j0 be its transition probabilities.

(a) Calculate _i = E(X₁|X₀ = i) as a function of (P_ij )i,j0. Then, calculate the covariance = Cov(X₀, X₁) as a function of (_i)i0 and (_i)i0.

(b) (10 marks) Let a (0, 1) and b (0, 1) be fixed. For any i 0, let p_i = bⁱ . Assume that:

_i = (1 ab)aⁱ p_i for any i 0

P_ij = (1 p_i) ^j1 p_i , for all j 1, and P_i0 = 0.

Use the result in part (a) to calculate = Cov(X₀, X₁) as a function of a and b.

Hint: If X is a geometric random variable with P(X = n) = (1 p) ⁿ¹p for n = 1, 2, 3, . . . and p (0, 1), then E(X) = 1/p.