Consider the Markov reward process described by the following state diagram and assume the agent is in state $0$

at time $$

$($also assume the discount rate is $=1$

$).$ A Markov reward process can be thought of as an MDP with only one action possible from each state $($denoted as action $0$

in the figure below$).$