6 Markov Chains:

Prove/Disprove Prove or disprove the following statements, using the definitions from the previous question.

(a) There exists an irreducible, finite Markov chain for which there exist initial distributions that converge to different distributions.

(b) There exists an irreducible, aperiodic, finite Markov chain for which P(X_n+1 = j | X_n = i) = 1 or 0 for all i, j.

(c) There exists an irreducible, non-aperiodic Markov chain for which P(X_n+1 = j | X_n = i) 6= 1 for all i, j.

(d) For an irreducible, non-aperiodic Markov chain, any initial distribution not equal to the invariant distribution does not converge to any distribution