Suppose there are four questions Q1, Q2, Q3, and Q4, which are associated with a reward of $100, $1; 000, $10; 000, and $50; 000, respectively. The rules are as follows: (a) when presented a question, the challenger (denoted by A) will have two choices, either to quit or to accept. If she chooses to quit, A will take all the money she has earned so far and leave. Assume that A knows in advance that she will pass Q1, Q2, Q3, and Q4 with respective probabilities 2=3, 1=2, 1=4, and 1=8. If she accepts and passes the challenges, she will be presented by the next question; if she accepts but fails, she will get $0 and the game ends; (b) A starts at the state Q1 and has $0 at hand; (c) The game will be over if A passes the last question Q4 and in that case, A will earn all the rewards over the four questions.

Consider a simple policy _^*, in which A that always accepts the challenge. Please compute the value function V^*(s). Here you should explicitly state the values V^*(s) for the four states s = Q1, Q2, Q3, and Q4. Lastly, compute an optimal policy _^* and the value function V^*. Again, please explicitly state the values V^*(s) for the four states s = Q1, Q2, Q3, and Q4.