3.4.6 Consider the Markov chain whose transition matrix is

where pCq D 1. Determine the mean time to reach state 4 starting from state 0.
That is, find E[TjX0 D 0], where T D minfn 0IXn D 4g.
Hint: Let vi D E[TjX0 D i] for i D 0;1; : : : ; 4. Establish equations for v0; v1; : : : ; v4 by using a first step analysis and the boundary condition v4 D 0.
Then, solve for v0.

Transcribed Image Text:

0 1 2 3 4 9 P00 0 0 0 09 1 9 0 P 00 P=200p0 3q000 p 400 0 0 0 0 1