Problem 5.1. Urn A initially has one white and one red marble, and urn B initially has one white and three red marbles. The marbles are repeatedly interchanged. In each step of the process one marble is selected from each urn at random and the two marbles selected are interchanged. Let the stochastic variable Y denote "configuration of the urns." Three configurations are possible: (1) Urn A-2 white balls, Urn B-4 red balls; (2) Urn A-one white and one red ball, Urn B-one white and three red balls; (3) Urn A-two red balls, Urn B-two white and two red balls. We shall denote these three realizations as y(1), y(2), and y(3), respectively.

(a) Compute the transition matrix, Q, and the conditional probability matrix, P(sols).

(b) Compute the probability vector, (P(s), at times, given the initial condition stated above. What is the probability that there are 2 red marbles in urn A after 2 steps? After many steps?

(c) Assume that the realization, y(n), equals n. Compute the first moment, (y(s)), and the autocorrelation function, (y(0)y(s)), for the same initial conditions as in part (b).