26. Consider a Markov chain with state space S. Let i, j ∈ S. We say that state j is accessible from state i in n steps if there is a path i = i1, i2, i3, . . . , in = j with i1, i2, . . . , in ∈ S and pimim+1 > 0, 1 ≤ m ≤ n − 1. Show that if S is finite having K states, and j is accessible from i, then j is accessible from i in K or fewer steps.

Hint: Use the Pigeonhole Principle: If n > K pigeons are placed intoK pigeonholes, then at least one pigeonhole is occupied by two or more pigeons.