answer all questions

Consider a Markov chain with state space S = {1, 2, ...,6} and transition matrix given by P = O O OAKW ONIN O ONIHO O O OO OO HNIH O O OAIHO O WITHOOOO WIN ONIHO O O (a) Find all stationary distributions for this chain (b) Find the probability that we ever reach state 1, starting from state 3 (c) Find the expected time that we first reach the set of states A = {2,5}, starting from state 3. (d) Starting from state 3, find the limiting proportion of time spent in state 1. (e) Starting from state 3, find the expected limiting proportion of time spent in state 1. (f) Repeat the above two questions, starting from state 4 instead of state 3