Consider a Markov chain associated withFOURumbrellas, instead of similar tothree, which was consid- ered in the notes. Assume that rains occur with the same probability,p,during the day, and this occurrence does not depend on the process,X={X(n) :n0}

1. Show transition probabilities for the state spaceS={1,2,3,4}.2. Derive stationary distribution for the number of umbrellas.

3. Evaluatelimiting expectation, limit(E[X(n)],)as n.