Hitting probabilities. Consider a Markov chain with finite state space S. Let a and b be two points in S, let  D Va ^ Vb, and let C D S  fa; bg. Suppose h.a/ D 1; h.b/ D 0, and for x 2 C we have h.x/ D X

y p.x; y/h.y/

(a) Show that h.Xn/ is a martingale.

(b) Conclude that if Px. < 1/ > 0 for all x 2 C, then h.x/ D Px.Va < Vb/ giving a proof of Theorem 1.27.