=+4. Demonstrate that a finite-state Markov chain is ergodic (irreducible and aperiodic) if and only if some power P n of the transition matrix P has all entries positive. (Hints: For sufficiency, show that if some power P n has all entries positive, then P n+1 has all entries positive.

For necessity, note that p

(r+s+t)

ij ≥ p

(r)

ik p

(s)

kk p

(t)

kj , and use the number theoretic fact that the set {s : p

(s)

kk > 0} contains all sufficiently large positive integers s if k is aperiodic. See Appendix A.1 for the requisite number theory.)