Simulation problem: An individual possesses r umbrellas that he employs in going from his home to office, and vice versa. If he is at home (the office) in the morning (in the evening) and it is raining, then he will take an umbrella with him to the office (home), provided there is one to be taken. If it is not raining, then he never takes an umbrella. Assume that, independent of the past, it rains in the morning (in the evening) with probability p. Let X, be the number of umbrellas at his home at the end of the nth day. Then, Xn defines a Markov chain. Let r = 3. a) Complete the following transition probability matrix P for this Markov chain. (For example, if Xn =0, then the probability that Xn+1 =0 is: Poo = prob(rain in the morning, no rain in the evening) + prob(no rain in the morning and evening) = p(1 -p) + (1 -p)?). (p(1 - p) + (1-p)2 ? P = ? p(1 - 0 D2 - +1-p b) For r = 3, use the formula given in class to confirm that the limiting probabilities are given by: if i = 0 M = if i = 1, . . . ,r where q = 1 - p. c) Let p = 0.5. Use Python to plot PB, i = 0, 1, 2, 3 for different values of n. Do Pig's converge as n grows? If yes, what is the relationship between lim, + PB, i = 0, 1, 2, 3 and #2? d) Repeat part c with p =0.1. If Pg's converge in both cases, which one converge faster