Consider the Bellman equation for deterministic policies and state-only rewards: VT (s) = R(s) + yT(s, (s), s')V (s') s' We often need to consider stochastic policies as well, which we denote by (als) instead of (s). T(als) specifies the probability of taking action a in state s. When the policy is deterministic, exactly one action a will have probability 1, so we overload notation and refer to that action as a = (s). Note: The output types are different; 7(als) outputs a probability, whereas T(s) outputs an action. A more general version of the Bellman equation can be derived for stochastic policies and reward functions depending on (s, a, s'): V" (s) = (a|s) T(s, a, s') [R(s, a, s') + yV (s')] a (a) Explain, in words, what the general version of the Bellman equation means. Additionally, show that it reduces to the simpler version when using deterministic policies 7(s) and state-only re- wards R(s).