3. (40 points) Caleb has decided to become a free lance programmer. After a week of working with a customer, he has 1 g probability of getting an offer to continue working with the same customer for the same salary of last week (b dollars per week), and with probability the customer will no longer require his service next week. Il he gets an offer, Caleb must decide whether to accept the offer or spend [ dollars per week on professional education. If he invests in his professional education, Caleb receives no revenue for the eurrent week, If he gets no offer, he always invests and spends one week doing professional education. After a week of not working, Caleb has a 1 p probability of being hired as a free lance programmer for 3b dollars per week or a p probability of being a free lance programmer for .86 dollars per week. *Assume Caleb always gets to work as a programmer after spending one week in profes- sional education, i.e., he will never spend two consecutive weeks in professional education. Update: Both of the following interpretations are acceptable: (a) Think of b as a constant number throughout the problem, 3b and 0.8h as well. (b) Think of b as the variable that changes every week, representing the most recent working salary. Questions: (a) (b) (e o (10 points) Assume Caleb received an offer to be a programmer at the beginning of week 1. Draw the decision tree for the first week, clearly labeling the probabilities as well as the rewards. For a full solution, indicate decisions made in week 1 as well as week 2's terminal conditions. (15 points) Formulate Caleb's decision problem as a Markov decision process, in which the objective is to maximize the total expected money he receives over the next one week. Assume he receives as terminal reward the same salary received at the beginning of week 2. You will receive full marks for clearly indicating the five basic components of this MDP. (15 points) Assume Caleb received an offer to be a programmer at the beginning of week 1 for b = $1000 and it costs [ = $500 per week to cover educational costs. Let p be 0.8 and g be 0.7. Should Caleb take the job offer or pursue professional education in the first week?\