For a stationary Discrete-Time Markov Chain (DTMC) we have, in standard form, P, P 2, and P3 P N and the associated matrix (I Q) . The state space for this system is the set 0.2 0.2 0.1 o o 0.1 0.2 0.2 o o 0.4 0.2 0.3 o o 0.2 o 0.2 1 o 0.1 0.4 0.2 o 1 0.3840 0.1360 0.3320 1 0.22 0.18 0.17 0.32 0.08 0.28 0.24 0.54 0.35 o 1 0.3420 0.6260 0.4370 0 1.4773 0.4545 0.3409 0.1 0.1 0.09 o o 0.4261 1.4773 0.4830 0.12 0.1 0.11 o o 0.0660 0.0580 0.0570 o o 0.0780 0.0660 0.0650 o o 0.1300 0.1140 0.1090 o o o o o o o o o o o o o o 1 o o o o o o 0.4886 0.2273 0.4205 o 0.5114 0.7727 0.5795 0 and (I 0.9659 0.6818 1.7614