A house has a simple thermostat that can be set at 1 to turn the furnace on, and 0 to turn it off. Potential changes in setting take effect every 10 minutes. If the thermostat is on 0 , in a 10 -minute period the room temperature will either stay the same or go down by a degree, with equal probability. If it is on 1 , the room temperature will go up by a degree with certainty in the next 10 -minute period. There is an energy cost of 1 cent for each 10 -minute period during which the furnace is on. There are also discomfort costs of 1.5 cents per degree for each 10 -minute period for each degree of room temperature difference between the current temperature and the ideal temperature of 68 . Assume that room temperature must be kept at all times between 65 and 71 , and that the thermostat must turn on when the temperature is 65 , and must turn off when it is 71 . How should the thermostat be programmed to operate? Formulate the problem as a Markov decision problem, write out the DP equation, and solve it using Mathematica for a time horizon of 50 minutes, using a terminal cost that penalizes differences between final temperature and 68 as described above.