38. To prove Equation (7.24), define the following notation: X{ time spent in state / on...

38. To prove Equation (7.24), define the following notation:

X{ Î time spent in state / on the y'th visit to this state;

Ni(m) = number of visits to state / in the first m transitions In terms of this notation, write expressions for

(a) the amount of time during the first m transitions that the process is in state /;

(b) the proportion of time during the first m transitions that the process is in state /.

Argue that, with probability 1,

(c) Ó Ô Ã 7 - Ô -+ßi as m oo j = l Ni(m)

(d) Ni(m)/m -» ð, as m -* oo.

Combine parts (a), (b), (c), and

(d) to prove Equation (7.24).