Justify or prove all answers.

(20 points) Consider the Markov chain associated to the random walk in the following undirected graph (the random walk starts at a node and at every step picks at random a node adjacent to the current node and moves to it): (a) Is this Markov chain irreducible? Is this Markov chain aperiodic? Is this Markov chain reversible? (b) Determine the invariant distribution. (c) Consider two independent walkers moving according to the random walk simulta neously, each with initial distribution equal to the invariant distribution. Let Sn be the number of times they meet at the same node in the rst n steps. Determine the expected value of Sn