A game consists of successively throwing 1 of 4 tetrahedrons, each of whose faces is numbered by a digit {1,2,3,4}, on a flat surface, according to the rules described below, and observing the digit of the face that falls down. The first tetrahedron produces the following probabilities for possibilities 1, 2, 3 or 4, respectively: {0, 1/3 , 1/3 , 1/3 }; the second tetrahedron: {1/4, 0, 1/2, 1/4}; the third: {1/4, 1/2 ,0 , 1/4}; and the fourth tetrahedron: {1/3, 1/3, 1/3, 0}. Rule: Launches are started by choosing one of the 4 tetrahedrons according to a given probability distribution. If a given move results in i, for a given i = 1, 2, 3, 4, then the next move is made with the i-th tetrahedron. 

(a) Describe the game in terms of a Markov Chain (Xn) in the proper state space, indicating the parameters of the chain. 

(b) Let (Yn) be the chain that records the result of the current move and those of the 2 subsequent throws, ie, Yn = (Xn,Xn+1,Xn+2), n ≥ 0. What is the space of (Yn) states? Is (Yn) Markovian? If so, describe the transition probabilities, and determine if it is irreducible? Justify. 

(c) Suppose X0 = 3. Find the expected number of moves from ́ı until you observe pattern 123 for the first time. Use, properly justifying, the relationship between expected return times and weights of the invariant distribution.