Consider a Markov chain with states 1, 2, 3, 4, 5, and suppose that we want to compute P(X4 = 2,X3 2,X2 2,X1 2|X0 = 1)

That is, we want the probability that, starting in state 1, the chain is in state 2 at time 4 and has never entered any of the states in the set A = {3, 4, 5}.

To compute this probability all we need to know are the transition probabilities P11, P12, P21, P22. So, suppose that P11 = 0.3 P12 = 0.3 P21 = 0.1 P22 = 0.2 Then we consider the Markov chain having states 1, 2, 3 (we are giving state A the name 3), and having the transition probability matrix Q as follows:

The desired probability is Q4 12. Raising Q to the power 4 yields the matrix

Transcribed Image Text:

0.3 0.3 0.3 0.4\ 0.1 0.2 0.7 0 0 1