Consider the grid-world given below and Pacman who is trying to learn the optimal policy. If an action Fesults in landing into one of the shaded states the corresponding reward is awarded during that transition. All shaded states are terminal states, i.e., the MDP terminates once arrived in a shaded state. The other states have the North, East, South, West actions available, which deterministically move Pacman to the corresponding neighboring state (or have Pacman stay in place if the action tries to move out of the grad). Assume the discount factor =0.5 and the Q-learuing rate =0.5 for all calculations. Pacman starts in state (1, 3), Hint: Q-values obtained by Q-learuing updates - Q(s,a)(1)Q(s,a)+(R(s,a,s)+maxQ(s,a)). (a) What is the value of optimal value function V at the following states: V(3,2)= V(2,2)=V(1,3)= (b) The agent starts from the top left corner and you are given the following episodes from runs of the agent through this grid-world. Each line is an Episode is a tuple containing (s,a,s,r). Hint: N, E, S, W refer to the moving directions. Using Q-learning updates, what the following Q-values after the above three episodes: Q((3,2),N)= Q((1,2),S)= Q((2,2),E)=