In Problem 5.5 in the MCS, a student is in one of three Markov states: 0:sleeping, 1:eating, or 2:reading. (To complete this assignment, you probably should review Problem 5.5 and its solution in the back of the MCS.) With respect to sleeping, Problem 5.5 says Sleeping is always followed by eating. Each time the student goes to sleep (takes a nap), the average nap time is 4 hours. Because sleeping (state 0) is a Markov state, at the end of each hour, the next state is given by the transition probabilities P00, P01 and P02. Next, since \sleeping is always followed by eating," we know P01 > 0 but P02 = 0. So, at the end of each hour, the student switches from sleeping to eating with probability P01. Hence the time (in hours) spent sleeping is a geometric random variable with success probability P01. This implies that the average time spent sleeping is 1=P01. Since the problem statement says that the average nap time is 4 hours, we learn that 1=P01 = 4, or P01 = 1=4. Since P2 j=0 P0j = 1, these facts imply P00 = 3=4. The time spent \sleeping" is also known as the sojourn time in the sleeping state. In general for any Markov chain, since the system departs state i with probability 1 - Pii, the sojourn time in any state i is a geometric random variable with expected value 1=(1 - Pii). This review of Problem 5.5 is just to introduce the following variation on Problem 5.5: Problem 1: Consider a student who is always either sleeping, eating or reading, such that If the student is eating, the student continues to eat for another hour with probability 0.1; otherwise, the student begins to read. Sleeping is always followed by eating. Each time the student goes to sleep (takes a nap), the nap time is exactly 4 hours. After an hour of reading, another hour of reading is four times more probable than an hour of eating. However, the probability of going to sleep equals the sum of the probabilities of reading or eating. Just as in Problem 5.5, the time step is still 1 hour. For this modified problem: (a) What is the Markov chain? Hint: The Markov chain needs to keep track of how many hours the student has slept (or how many hours of sleep the student has remaining.) (b) What are the stationary probabilities? (c) After a very long time, what is the probability the student is sleeping? Once you solve this problem, now consider the following generalization: