* In problem 1 and 2 , we consider a two-state Markov decision process (S,A,P,R,) where S={1,2} is a state space, A={a1,a2} is an action space, P is a transition probability matrix such that P(11,a1)=0.9,P(21,a1)=0.1,P(12,a1)=0,P(22,a1)=1,P(11,a2)=P(21,a2)=0.5,P(12,a2)=0.2,P(22,a2)=0.8, R is the reward such that R(1,a1)=1,R(1,a2)=3,R(2,a1)=1,R(2,a2)=0 and [0,1) is a discounted factor. =0.9