Optimal Policy and Value Function (deep learning)

Consider a simple 2-state MDP (Markov Decision Process) shown in Figure 1, S = {S1, S2}. From each state, there are two available actions A = {stay, go}. The reward received for taking an action at a state is shown in the figure for each (s, a) pair. Also, taking action go from state S2 ends the episode. All transitions are deterministic.