Three Independent Markov Chains

Alice has 3 dogs: Bishi, Mishi and Rishi. Each dog independently explores the neighborhood. Let X_n,Y_n,Z_n be the locations at time n of Bishi, Mishi and Rishi respectively, where time is assumed to be discrete and the number of possible locations is a finite number M. Their paths X₀,X₁,X₂,... ; Y₀,Y₁,Y₂,... and Z₀,Z₁,Z₂,... are independent Markov chains with the same stationary distribution s. Each dog starts out at a random location generated according to the stationary distribution.

(a) Let state 0 be home (so s₀ is the stationary probability of the home state). Find the expected number of times that Rishi is at home, up to time 24, i.e., the expected number of how many of X₀,X₁,...,X₂₄ are in state 0 (in terms of s₀).

(b) If we want to track all 3 dogs simultaneously, we need to consider the vector of positions, (X_n,Y_n,Z_n). There are M³ possible values for this vector; assume that each is assigned a number from 1 to M³, e.g., if M = 2 we could encode the states (0, 0, 0), (0, 0, 1), (0, 1, 0), ..., (1, 1, 1) as 1, 2, 3, ..., 8 respectively. Let W_n be the number between 1 and M³ representing (X_n, Y_n, Z_n). Determine whether W₀, W₁, . . . is a Markov chain.

(c) Given that all 3 dogs start at home at time 0, find the expected time it will take for all 3 to be at home again at the same time.