Markov chain. 8 Consider an irreducible Markov chain on a finite state space, such that the transition

matrix is symmetric (pij = pji for all i, j ? S). Find the stationary distribution.

9 Markov chains are named for Russian mathematician A. A. Markov, who in the early

twentieth century examined the sequence of vowels and consonants in the 1833 poem

Eugene Onegin by Alexander Pushkin. He empirically verified the Markov property and

found that a vowel was followed by a consonant 87% of the time and a consonant was

followed by a vowel 66% of the time. (a) Give the transition graph and the transition

matrix. (b) If the first letter is a vowel, what is the probability that the third is also a

vowel? (c) What are the proportions of vowels and consonants in the text?

10 A text is such that a vowel is followed by a consonant 80% of the time and a consonant

is followed by a vowel 50% of the time. In the following cases, how should you guess in

order to maximize your probability to guess correctly: (a) a letter is chosen at random,

(b) a letter is chosen at random and the next letter in the text is recorded, (c) five letters

are chosen at random with replacement, (d) a sequence of five consecutive letters is

chosen at random?

11 Consider a text composed of consonants, vowels, blank spaces, and punctuation marks.

When a letter is followed by another letter, which happens 80% of the time, the probabilities are as in the previous problem. If a letter is not followed by a letter, it is followed

by a blank space 90% of the time. A punctuation mark is always followed by a blank

space, and a blank space is equally likely to be followed by a vowel or a consonant. (a)

State the transition matrix and find the stationary distribution. (b) If a symbol is chosen

at random and turns out to be a punctuation mark, what is the expected number of blank

spaces before the next punctuation mark? (c) If this is a literary text in English, what in

the model do you find unrealistic?

12 Customers arrive at an ATM where there is room for three customers to wait in line.

Customers arrive alone with probability 2

3

and in pairs with probability 1

3

(but only one

can be served at a time). If both cannot join, they both leave. Call a completed service or

an arrival an "event," and let the state be the number of customers in the system (serviced

and waiting) immediately after an event. Suppose that an event is equally likely to be an

arrival or a completed service. (a) State the transition graph and transition matrix and

find the stationary distribution. (b) If a customer arrives, what is the probability that

he finds the system empty? Full? (c) If the system is empty, the time until it is empty

again is called a "busy period." During a busy period, what is the expected number of

times that the system is full?

13 Show that a limit distribution is a stationary distribution. The case of finite S is easier,

so you may assume this

1. a. Find the MGF of a standard normal random variable and its region of convergence. b. Using an affine transform on a standard normal random variable, find the MGF of a Gaussian random variable with expected value ux and standard deviation ox. c. Find the MGF of a sum of n i.i.d. Gaussian random variables. d. Using your results in parts (b) and (c) find the PDF of a sum of n i.i.d. Gaussian random variables.Which of the following is NOT true? O Uncorrelated Gaussian random variables will always have a diagonal covariance matrix. O b ) Independent Gaussian random variables will always have a covariance matrix that has a determinant of zero. Gaussian random variables with a diagonal covariance matrix are uncorrelated. O d) Uncorrelated Gaussian random variables are also independent.. Let X and Y be independent stande Gaussian random variables. Let Z = 2X Y. Is Z a Gaussian random variable? Find the mean and variance of Z. Find the density of Z. Consider the random vector V = [g '11] [g] . Is V a Gaussian random vector? Find the density of V. Does V have independent coordinates? two zero-mean real-valued Gaussian random variables X and Y are called jointly Gaussian if and only if their joint density is 1 fxy(x, y) = 27 \\ det E exp -7 (x, y) 2-1 (x, y) T ) (2.44) where (for zero-mean random vectors ) the so-called covariance matrix E is E = E [(X, Y) (x, Y)] =( OXY (2.45) OXY (a) Show that if X and Y are zero-mean jointly Gaussian random variables, then X is a zero-mean Gaussian random variable, and so is Y. (b) Show that if X and Y are independent zero-mean Gaussian random variables, then X and Y are zero-mean jointly Gaussian random variables. (c) However, if X and Y are Gaussian random variables but not independent, then X and Y are not necessarily jointly Gaussian. Give an example where X and Y are Gaussian random variables, yet they are not jointly Gaussian. (d) Let X and Y be independent Gaussian random variables with zero mean and variance o'x and ov, respectively. Find the probability density function of Z = X+Y. Observe that no computation is required if we use the definition of jointly Gaussian random variables