5 Consider the success run chain in Example 8.2.16. Suppose that the chain has been

running for a while and is currently in state 10. (a) What is the expected number of

steps until the chain is back at state 10? (b) What is the expected number of times the

chain visits state 9 before it is back at 10?

16 Consider a version of the success run chain in Example 8.2.16 where we disregard

sequences of consecutive tails, in the sense that for example T, T T, T T T, and so on,

all simply count as T. Describe this as a Markov chain and examine it in terms of

irreducibility, recurrence, and periodicity. Find the stationary distribution and compare

with Example 8.2.16. Is it the limit distribution?

17 Reversibility. Consider an ergodic Markov chain, observed at a late timepoint n. If

we look at the chain backward, we have the backward transition probability qij =

P(Xn?1 = j|Xn = i). (a) Express qij in terms of the forward transition probabilities

and the stationary distribution ?. (b)If the forward and backward transition probabilities

are equal, the chain is called reversible. Show that this occurs if and only if ?ipij =

?jpji for all states i, j (this identity is usually taken as the definition of reversibility).

(c) Show that if a probability distribution ? satisfies the equation ?ipij = ?jpji for all

i, j, then ? is stationary.

18 The intuition behind reversibility is that if we are given a sequence of consecutive states

under stationary conditions, there is no way to decide whether the states are given in

forward or backward time. Consider the ON/OFF system in Example 8.2.4; use the

definition in the previous problem to show that it is reversible and explain intuitively.

19 For which values of p is the following matrix the transition matrix of a reversible Markov

chain? Explain intuitively.

P =

0 p 1 ? p

1 ? p 0 p

p 1 ? p 0

!

20 Ehrenfest model of diffusion. Consider two containers containing a total of N gas

molecules, connected by a narrow aperture. Each time unit, one of the N molecules is

chosen at random to pass through the aperture from one container to the other. Let Xn

be the number of molecules in the first container. (a) Find the transition probabilities

for the Markov chain {Xn}. (b) Argue intuitively why the chain is reversible and why

the stationary distribution is a certain binomial distribution. Then use Problem 17 to

show that it is indeed the stationary distribution. (c) Is the stationary distribution also

the limit distribution?

21 Consider an irreducible and positive recurrent Markov chain with stationary distribution

? and let g : S ? R be a real-valued function on the state space. It can be shown that

1

n

Xn

k=1

g(Xk)

P?

X

j?S

g(j)?j

for any initial distribution, where we recall convergence in probability from Section 4.2.

This result is reminiscent of the law of large numbers, but the summands are not i.i.d.

We have mentioned that the interpretation of the stationary distribution is the long-term

proportion of time spent in each state. Show how a particular choice of the function g

above gives this interpretation (note that we do not assume aperiodicity)

2. (10 pts) Picking stickers. Envelope 1 contains 51 Stat 134 stickers and d1 Stat 140 stickers. Envelope 2 contains 52 Stat 134 stickers and d2 Stat 140 stickers. Your friend chooses one envelope uniformly at random and hands it to you. You pick one sticker out uniformly at random and it is a Stat 134 sticker. Without replacing the sticker you just picked, what is the chance that the next sticker you pick (also uniformly at random) from the same envelope is a Stat 140 sticker? Suppose you want to estimate the percentage of stat students who passed the departmental final exam. You take a sample of 50 stat students and grade their finals first to get an estimate Match the following: E] whether or not they passed the final @percentage of all stat students who passed the departmental final exam E] all stat students E] 50 stat students E] a stat student E] the percentage of 50 stat students who passed the departmental final exam . statistic p- a! sample . variable individual parameter #9?\" population 2. Suppose 42%% of Ph.D.s in statistics/biostatistics are earned by women and a random sample 36 statistics/biostatistics Ph.Dis is obtained. (You may want to use technology to compute some of the probabilities.) a. What's the probability none of the 36 statistics/biostatistics Ph.D.s are women? b. What's the probability at least 1 of the 36 statistics/biostatistics Ph.D.s is a woman? c. How many women do you expect in the sample of the 35 statistics/biostatistics Ph.D.s? d. What's the probability fewer than half of the 36 statistics/biostatistics Ph.D.s are women?6. At a large university, the probability that a student takes statistics and a foreign language in the same semester is 0.08. The probability that a student takes statistics is 0.2. If taking statistics and taking a foreign language are independent events, find the probability that a student takes statistics or a foreign language in the same semester