q1 help
Solution. Let X, be the color of the ball in the not draw, n 2 1. The state space is { R, W. B), where e.g., "R" denotes Red. The reason why this is a Markov chain is that: (1) the number of states is finite, i.e., the random variable X, is discrete, and (ii) the Markov property holds since the future evolution of the states (the color of the ball drawn), is independent of the outcome of past draws if we condition on the current color of the ball (the current state). In other words, it does not matter what colors we obtained in the past in order to determine the future transition probabilities, conditional on knowing the current ball color. Let us calculate the entries of the transition probability matrix row by row. The first row corresponds to obtaining a red ball at this draw, the second corresponds to drawing a white ball, and the third a blue ball. We calculate all the possible transition probabilities as follows: PRR = P(Xn+1 = RX, = R) = 1/5 PRW = P(X+1 = WIX, = R) = 0 PRB = P(Xn+1 = B|X, = R) = 4/5. This is because if we draw a red ball at this turn (irrespective of the urn from which we draw that ball), then we know that the next ball will be drawn from the red urn as well. Since there is a total of 5 balls in that urn, where 1 ball is red and 4 balls are blue, and no balls are white, then it must be that PRA = 1/5, PR,W = 0, and PR.B = 4/5. Similarly, PWR = P(Xn+1 = RIX, = W) =2/7 Pww = P(Xn+1 = W|X, = W) = 3/7 PW.B = P(Xn+1 = BIX, = W) =2/7, since conditional on drawing a white ball in this round, we must draw next from the white urn. However, in the white urn there are a total of 7 balls in the white urn, of which 2 are red, 3 are white, and 2 are blue. Arguing similarly, we get that: PBR = P(X,+1 = RIX. = B) = 3/9 Pow = P(X,+1 = W|X, = B) = 4/9 PBB = P(Xn+1 = B|X, = B) = 2/9. Combining everything, the transition probability matrix is given by: [1/5 0 4/5] P = 2/7 3/7 2/7 3/9 4/9 2/9 We can get the long-run proportion of the 3 colors drawn by solving the system of equations: TR 3 4 FTW + -KB 4 2 TRA=TW+ TR+B + TW 1. Solving this system of equations, we get that: 25 28 36 TW= 89 89 89 which give the relative proportions that the drawn balls are red, white, and blue respectively.1. [20 points] We let { S. : n 2 1} be a sequence of independent exponential random variables, each having mean E[S.,] - 2. We define for n 2 4, the random variable T, = 1 if S, = min{ Sn, Sm-1, S.-2, Sn-3); otherwise, we define T. = 0. Also let Ti = T2 - 73 - 0. (a) [10 points] Determine the mean and variance of Th for n 2 4. (b) [10 points] Determine the covariance Cov(7%, In+1) for n 2 4
