(c) Consider the following MDP where every state transition receives a reward of 1 and discount factor is =1 : Figure 1: A simple MDP with 2 non-terminal states A and B Then v(X) of a state X{A,B} essentially represents the expected number of steps a policy needs to reach the terminal state from state X. The goal is to maximize v(X), which is equivalent to minimize the number of steps before reaching the terminal state. (c.1) (1%/0.5%) What is the expected number of steps a equiprobable random policy (taking all allowed actions equally likely) needs to reach the terminal state from state A ? (In other words, what is v(A) when is a uniform policy?) (c.2) (1%/0.5%) Describe the optimal -soft policy for this MDP when is a small positive number. (c.3) (1%/1%) When =0.1, what is the expected number of steps the optimal -soft policy needs to reach the terminal state from state A ? (In other words, what is v(A) when the optimal 0.1-soft policy?)