1. The following figure shows a Grid World problem, in which (1) the grey and solid block represents a barrier(2) the robot can get the reward of +1 and -1 when it goes to the position of (4, 3) or (4, 2), while the rewards for any other positions are 0.

Question: Use the discounted reward with the discount factor of 0.9 to compute Q values in this problem and determine the best actions at each position ( dont need to consider Q values and actions at (43) and (42) ). [10 points]

2. Determine whether I like a document or not according to the word I included in the document. The previous statistical data indicate that only 20% documents are liked by me. The probability of including I in the documents which I like is 98%, and the probability of including I in the documents which I dont like is 32%. Now there is a document which includes I. Please infer whether I like this document according to Bayes decision rule. The computation process is required. [10 points]