4.6. Consider the Markov chain whose transition matrix is

where p + q = 1. Determine the mean time to reach state 4 starting from state 0. That is, find E[TIXO = 0] where T = min{n ? 0; X,, = 4}.
Hint: Let v, = E[TIXO = i] for i = 0, 1, . . . , 4. Establish equations for v0, v...... v4 by using a first step analysis and the boundary condition v4 = 0. Then solve for v0.

Transcribed Image Text:

P = 0 1 2 3 4 09 9 P 00 0 PO 19 0 P 0 0 2900 P 0 3 6 000 4 || 0 0 0 0 0