For a stationary Discrete-Time Markov Chain (DTMC) we have, in standard form, P, P2, and PS, P and the associated matrix (I - Q) . The state space for this system is the set (1, 2, 3, 4, 5}. 0.2 0.1 0.4 0.2 0.1 0.1 0.12 0.22 0.32 0.24 0.2 0.2 0.2 0 0.4 0.1 0.1 0.18 0.08 0.54 P = 0.1 0.2 0.3 0.2 0.2 P2 = 0.09 0.11 0.17 0.28 0.35 0 0 0 0 0 0 0 0.0660 0.0780 0.1300 0.3840 0.3420 0 0 0 0.4886 0.5114 0.0580 0.0660 0.1140 0.1360 0.6260 0 0 0 0.2273 0.7727 P = 0.0570 0.0650 0.1090 0.3320 0.4370 , P(0) = 0 0 0 0.4205 0.5795 0 0 0 0 0 0 0 0 0 0 0 1.4773 0.4261 0.9659 and (I - Q)-1 = 0.4545 1.4773 0.6818 0.3409 0.4830 1.7614 1-f the initial state at time 0 is 3, and three thousand transitions later the state is 1, what is the probability that the process enters a transient state at epoch 3002? 2-the process starts in State 1, what is the expected number of visits to State 3 in the first 3 epochs? 3-assuming that at epoch 3, the system is in State 3, compute the probability that the system ever visits State 4 in two ways? 4-What is the mean number of epochs spent in State 2, per visit to State 2? 5-Will solving the set of simultaneous linear equations n = n xP, along with the normalizing equation n x1 = 1 provide a meaningful solution? If so, specify the algorithm and offer an interpretation for the resulting vector , and if not, then explain why not. Here 1 is a 5 x1 column vector. 6-the process starts at time zero in State 1, what is the mean time until the process enters its terminating state? 7-Since this P is already written in standard form, specify the R submatrix (actual values are necessary here), and interpret the meaning of R. 8-n the long-run, what is the variance of the time until the first visit to State 4 from initial state State 1, if we know that State 4 will definitely be visited? Write out the steps of your algorithm and then compute the exact number using MATLAB and any MATLAB code that I provided in the class