31. An individual possesses r umbrellas which he employs in going from his home to office, and vice versa. If he is at home (the office) at the beginning (end) of a day and it is raining, then he will take an umbrella with him to the office (home), provided there is one to be taken. If it is not raining, then he never takes an umbrella. Assume that, independent of the past, it rains at the beginning (end) of a day with probability p.

(i) Define a Markov chain with r + 1 states which will help us to determine the proportion of time that our man gets wet. (Note: he gets wet if it is raining, and all umbrellas are at his other location.)

(ii) Show that the limiting probabilities are given by if / = 0 r + q 1 where q = 1 - ñ
if / = 1, . . . , r (iii) What fraction of time does our man get wet?
(iv) When r = 3, what value of ñ maximizes the fraction of time he gets wet?