Expectations of hitting times. Consider a Markov chain state space S. Let A  S and suppose that C D S  A is a finite set. Let VA D minfn  0 W Xn 2 Ag be the time of the first visit to A. Suppose that g.x/ D 0 for x 2 A, while for x 2 C we have g.x/ D 1 C X

y p.x; y/g.y/

(a) Show that g.XVA^n/ C .VA ^ n/ is a martingale.

(b) Conclude that if Px.VA <

1/ > 0 for all x 2 C then g.x/ D ExVA, giving a proof of Theorem 1.28.