An individual possesses r umbrellas that he employs in going from his home to office, and vice versa. If he is at home (the office) at the beginning (end) of a day and it is raining, then he will take an umbrella with him to the office (home), provided there is one to be taken. If it is not raining, then he never takes an umbrella.

Assume that, independent of the past, it rains at the beginning (end) of a day with probability p.

(a) Define a Markov chain with r + 1 states, which will help us to determine the proportion of time that our man gets wet. (Note: He gets wet if it is raining, and all umbrellas are at his other location.)

(b) Show that the limiting probabilities are given by