Consider the 43 environment and the policy 2 discussed in the lecture. Assume that the discount factor =1 and the transition is deterministic and R(s,a)=0.04 for non-terminals. Q.3) Solving Bellman Equation /10 Calculate U2(s) for every s using the Bellman Equation and the reward function discussed in class: U2(s)=R(s)+nP(ss,2(s))U2(s) Q.4) Better Actions: Policy Iteration /5 What would 2(1,1) be if using the U2 calculated in Q.3), one step of the following update rule is applied on (1,1) : 2(s)aA(s)argmax(R(s,a)+sP(ss,a)U2(s)) where A(s) is the set of actions available to the state s