For a stationary Discrete-Time Markov Chain (DTMC) we have, in standard form, P, P2, and P3, Po and the associated matrix (I - Q). The state space for this system is the set {1, 2, 3, 4, 5}. 0.2 0.1 0.4 0.2 0.1 0.1 0.12 0.22 0.32 0.24 0.2 0.2 0.2 0 0.4 0.1 0.1 0.18 0.08 0.54 P = 0.1 0.2 0.3 0.2 0.2 P2 = 0.09 0.11 0.17 0.28 0.35 0 0 0 1 0 O O O O O 0 0 O O 0.0660 0.0780 0.1300 0.3840 0.3420 0 0 0.4886 0.5114 0.0580 0.0660 0.1140 0.1360 0.6260 0 0 0.2273 0.7727 P3 = 0.0570 0.0650 0.1090 0.3320 0.4370 , P(0o) = 0 0 0.4205 0.5795 0 0 0 OH OH O O O 1.4773 0.4261 0.9659 and (I - Q)-1 = 0.4545 1.4773 0.6818 0.3409 0.4830 1.7614