Markov inequality

4. Consider a discrete-time Markov chain with the following probability transition matrix 0 0 0 0 7 1-3-y P = 1-I-VVO T 0 0 1 0 Is it possible to choose values for a and y so that the Markov chain has the following properties? In each case, state the values of a and y, or give a brief reason why it is not possible. (a) The Markov chain has period 2. [2) (b) The Markov chain is reducible. (c) The Markov chain has at least one transient state. UNN (d) The Markov chain has invariant distribution (1/4, 1/4, 1/4, 1/4).Markov inequality For a random variable X 2 0 with mean / > 0, and any numbert > 0: P (X > D). Note that the Markov inequality is restricted to non-negative random variables. Chebyshev inequality For a random variable X with (finite) mean / and variance of, and for any number t 2 0, P(X -/| 20) . Remark: When Markov inequality is applied to (X - ()", we obtain Chebyshev's inequality. Markov inequality is also used in the proof of Hoeffding's inequality. Hoeffding versus Chebyshev 4 points possible (graded) Let X1, X2, ..., X'n " Unif (0, b) be ni.i.d. uniform random variables on the interval [0, b] for some positive b. Suppose n is small (i.e. 7