4. The demand for a certain product can be either high or low each month. In any particular month, the demand depends only on the demand in the last two months. This month's demand depends on previous demand in the following ways: If the demand was high for the past two months, it will be also high this month with a chance 2/3 and low with a chance 1/3. If the demand was low for the past two months, it will be high this month with a chance 1/6 and low with a chance 5/6. If the demand last month was different from the demand the month before, then it is equally likely to be high or low this month. (a) Model this stochastic process as a time-homogeneous Markov chain with 4 states. Write the transition matrix. (b) Draw the state decision diagram. Is this Markov chain irreducible and ergodic? Explain. (c) Assume the demand was low in January and high in February. What is the chance of the demand being high in April? (d) Compute the steady state probabilities for this Markov chain. (e) The profit from selling this product is on average 12,000 in months with high demand and 8000 in months with low demand. What is the expected yearly profit in the long term?