MAST30001 Stochastic Modelling Assignment 1 Please complete the Plagiarism Declaration Form on LMS. Don't forget to staple your solutions and to print your name, student ID, and the subject name and code on the first page (not doing so will forfeit marks). The submission deadline is 3pm, Friday Sept 1 in the appropriate assignment box at the north end of Peter Hall building (near Wilson Lab). Marks may be lost where answers are not clear and concise (or where lacking in explanation). 1. A DTMC with state space S = {1, 2, 3, 4} has the following transition matrix 1/2 1/2 0 0 1/2 0 1/2 0 0 1/2 0 1/2 . 0 0 1 0 [10 Marks] (a) (b) (c) (d) (e) (f) (g) Does the chain have any absorbing state(s)? Is the chain irreducible? Is the chain periodic? If so, what is the period? Is the chain transient, null-recurrent, or positive recurrent? Is the process reversible? Find the long run proportion of time spent in state 4. If the initial distribution is uniform, does the limiting distribution exist? If so, find it. (h) Starting from state 2, find the probability of hitting state 1 before state 4. 2. Suppose we roll a fair six sided die repeatedly. [4 Marks] (a) Find the expected number of rolls required to see 3 sixes appear in succession. (b) Find the expected number of rolls required to see 3 of the same number in succession. 3. Batman chases the Joker around the vertices of a square. At each time step, Batman steps clockwise with probability p (0, 1), and anticlockwise with probability 1 p, while the Joker stays where he is with probability q [0, 1], and steps clockwise with probability (1 q)s and anticlockwise with probability (1 q)(1 s), where s (0, 1). Batman catches the Joker if they reach the same vertex at the same time. All steps are taken independently of previous steps. Starting from opposite corners of the square: [6 Marks] (a) what is the expected time until Batman catches the Joker when q = 1? (b) if q = 0, find the expected time until Batman catches the Joker, and for fixed p, find the maximum possible value for this quantity (i.e. optimise over s). (c) When p = 1/2, find the expected time until Batman catches the Joker