10. Markov chain Monte Carlo. We wish to simulate a discrete random variable Z with mass function satisfying P(Z = i ) i ,

10. Markov chain Monte Carlo. We wish to simulate a discrete random variable Z with mass function satisfying P(Z = i ) ∝ πi , for i ∈ S and S countable. Let X be an irreducible Markov chain with state space S and transition matrix P = (pi, j ). Let Q = (qi, j ) be given by qi, j =





min



pi, j , (πj /πi )p j,i if i 6= j, 1 −

X j : j 6=i qi, j if i = j.