4 Referring to Problem 1, suppose that tomorrow’s Smalltown weather depends on the last two days of Smalltown weather, as follows: (1) If the last two days have been sunny, then 95% of the time, tomorrow will be sunny.

(2) If yesterday was cloudy and today is sunny, then 70% of the time, tomorrow will be sunny. (3) If yesterday was sunny and today is cloudy, then 60% of the time, tomorrow will be cloudy. (4) If the last two days have been cloudy, then 80% of the time, tomorrow will be cloudy.

Using this information, model Smalltown’s weather as a Markov chain. If tomorrow’s weather depended on the last three days of Smalltown weather, how many states will be needed to model Smalltown’s weather as a Markov chain?

(Note: The approach used in this problem can be used to model a discrete-time stochastic process as a Markov chain even if Xt1 depends on states prior to Xt, such as Xt1 in the current example.)