3. Question: Rock around the clock: [15 Marks] Let (X,)n>o be a Markov chain defined on a state space I = {1, 2,3, ..., 12}. Consider this to represent a clock face. At each step the Markov chain will move to one of the adjacent neighbouring hours. In fact the stochastic matrix is as follows: if j = i+1 mod 12 Puj = q = (1-p) if j = i -1 mod 12 otherwise where p E (0, 1) Furthermore, suppose that the chain is started in state 6, i.e. Xo = 6. (a) What the probability that the chain hits state 11 in finite time? (b) What is the probability that the chain hits state 11 before state 1? (c) What is the probability that the chain first enters the state 12 by making a move from state 11? (d) What is the expected hitting time of state 12 starting from state 6? What value of p maximises the expected hitting time? (e) Suppose that 2 friends, Jack and Jill, play a game based on this Markov chain. Each game the chain is started at state 6. If it hits state 1 before state 11 Jill wins and Jack pays Jill Er. If it hits state 11 before state 1 then Jill pays Jack Ly. Work out the expected winnings for Jill on a single game as a function of p, I, y. (f) Suppose that r=10 and p = 1/3. Calculate the value of y that makes this fair game in the my ertel winnings a