Here we will look at a discrete Markov chain with infinite states i = 0, 1, . . .

Let p1,j = P(go from state i to state j in one step)= 1/(i+2) for j = 0,1,..i+ 1 (for example, for i=0 we have p(0,1) = p(0,2) = 1/2)

a) Find the steady-state probabilities.

b) Find the expected value.