undefined

Utility, Policy, and Their Calculation Consider the 4 x 3 environment discussed in the lecture. Let 7 be the following policy Right if you can; Else, UP if you can; Otherwise, Left; For example, A(1,1) = Right, 7(1, 2) = Up, and 7(4,1)= Left Assume that the discount factor 7= 1 and the transition is deterministic 3 +1 -1 2 i.e, P(s'|s, a) is either 0 or 1. E.g., P((2,1)|(1,1), Right) = 1, while P((1, 2) (1,1), Right) = 0 1 2 3 Q.3) Value Iteration / 10 Calculate U"(s) for every s using the Bellman Equation U* (s) = R(8) +-P(s'|s, 7(s))U*(s') (For example, U*(3,3) = 1 and U"(3, 2) = -1.) Q.4) Policy Iteration / 5 What would 1(1, 1) be if using the U* calculated in Q.3), one step of the following policy update rule is applied on (1, 1): a(s) + ars max ((8,0) + P(819 , a)0* (87) GEA(s) where A(s) is the set of actions available to the state s. Utility, Policy, and Their Calculation Consider the 4 x 3 environment discussed in the lecture. Let 7 be the following policy Right if you can; Else, UP if you can; Otherwise, Left; For example, A(1,1) = Right, 7(1, 2) = Up, and 7(4,1)= Left Assume that the discount factor 7= 1 and the transition is deterministic 3 +1 -1 2 i.e, P(s'|s, a) is either 0 or 1. E.g., P((2,1)|(1,1), Right) = 1, while P((1, 2) (1,1), Right) = 0 1 2 3 Q.3) Value Iteration / 10 Calculate U"(s) for every s using the Bellman Equation U* (s) = R(8) +-P(s'|s, 7(s))U*(s') (For example, U*(3,3) = 1 and U"(3, 2) = -1.) Q.4) Policy Iteration / 5 What would 1(1, 1) be if using the U* calculated in Q.3), one step of the following policy update rule is applied on (1, 1): a(s) + ars max ((8,0) + P(819 , a)0* (87) GEA(s) where A(s) is the set of actions available to the state s