10. Consider three r.v.'s X1, X2, Xs with finite sample space X; e {1,.... K}. The sequence Xi, i = 1, 2, 3, is called a Markov chain if Pr(X3 = i | X2 = j, X1 = k) = Pr(X3 = i | X2 = j) = Pr(X2 = i [ X1 = j). (1) That is, Pr(Xs = i | X2 = j) = Pji does not depend on X1, and it is the same as Pr(X2 = i | X1 = j). We call Py the transition probabilities. Let P = [P;] denote the (K x K) matrix of transition probabilties. 10a. Show Pr(Xs = i | X1 = j) = [P?], (the (j, i) element of P2). Hint: Use the law of total probability with Ex = {X2 = k}. Recall that any result for probabili- ties, like the law of total probability, is also true for conditional probabilities, like Pr(. | X] = j). Let LHS denote Pr(Xs = i | X1 = j). Which of the following arguments shows the claim? (a) LHS = ER, Pr(X2 = k | X1 = j) Pr(Xs = i | X2 = k, X1 = j) = Ex Pjk Pki = [P2]ji (b) LHS = \\ K Ek= Pr(X3 = k | X1 = j) = [P' ]ji (c) LHS = Pr(X2 = i | X1 = j) . Pr(Xs = i | X2 = j) = [P']ij (d) LHS = Pr(X2 = i | X1 = j) . Pr(Xs = i | X1 = j) = [P= ]ij (e) none of these Let q1 = Pr(X1 = i) denote the marginal probabilities for X1, 91 = (911, ..., qix) (a (1 x K) row vector), and similarly for q, and 93- 10b. Show q2 = q, P and 43 = q p2 Which of the following arguments shows the claim q, = q, P? (a) q2 = Pr(X2 = i) = Pr(X1 = j, X2 = 1)/Pr(X; = j) = Pr(X2 = i | X1 = j) = Pji (b) q2 = ); px, () Pxax, (i | X1 = j) = E, nj Pai (c) q2 = Pr(X2 = i) = Pr(X1 = j | X2 = 1)/[Pr(X2 = i)Pr(X2 = i | X1 = j)] (d) q2 = Pr(X2 = i) = _, Pr(X1 = j, X2 = i) (e) none of these 10c. Let * = (#1, .... *x)' be a probability vector (i.e., #x 2 0 and _ xx = 1) with The Pkj = Nj Pike for any pair of states j and k. If q1 = n, show that 92 = 93 = * as well (# is called an "equilibrium distribution" ). (a) By the law of total probability q2 = ), ; Pji = ), ";Puj = "; ), Paj = Ni (b) By Bayes' theorem 42: = . TK Pet it , Pki (c) q1 is a probability vector and P is a stochastic matrix # 92 =43 = 91. (d) By definition of conditional probability Pry = PRock,4351 = JAPLi = Pink = Ti. Pr(X1=k) IT k TK (e) none of these Similar definitions are used for a sequence of random variables Xt, t = 1,2, 3. .... See chapter 7 in the book. Definition 7.1 is the general version of (1); equation (7.1) is similar to (b) above; Definition 7.8 defines an equilibrium distribution; and Theorem 7.10 is (c)