Consider the Markov chain with transition graph given in Figure 3.

17, starting in state 1.

(a) Construct a computer program to simulate the Markov chain, and show a realization for \(N=100\) steps.

(b) Compute the limiting probabilities that the Markov chain is in state \(1,2, \ldots, 6\), by solving the global balance equations (C.42).

(c) Verify that the exact limiting probabilities correspond to the average fraction of times that the Markov process visits states \(1,2, \ldots, 6\), for a large number of steps \(N\).

Transcribed Image Text:

2 0.5 3 0.5 0.2 Start 0.7 0.3 0.1 0.9 0.2 0.3 0.8 5 0.5 Figure 3.17: The transition graph for the Markov chain {X+, t = 0,1,2....}.