In the following questions, assume that the mathematical model of an MDP (S,A,ps,r,) is given in the following form. - State set S is represented by integers S={1,,n}, - Action set A is represented by integers A={1,,m}, - State transition probabilities ps are given by a 3D array P in the form pijk=ps(sjsi,ak), - Rewards r are given by a 3D array R in the form rijk=r(si,Sj,ak). Note that since the sets S and R are composed of consecutive integers starting from 1 , it is enough to known m and n to define these sets. m and n can be easily obtained from the dimensions of the matrix P or R. Consider a finite horizon LQR problem defined by the tuple (A,B,Q,R,,T). A) Write a MATLAB function that takes the data of problem and returns the parameters of the optimal value function (Pt,ct) and the optimal control gains (Kt) which depends on time. Use cell arrays for return values such that each cell will contain data corresponding to a time t. B) For T=30, use your function to find optimal value function parameters and control gains of the inverted pendulum problem whose data was given below x1(t)x2(t)x3(t)x4(t)=10000.11000.0050.10151.04530.91360.00020.0050.10151.0453x1(t1)x2(t1)x3(t1)x4(t1)+0.00050.010.00050.0102f(t) Take Q=2000020000100003 and R=4 C) Write a MATLAB function that takes controller gains, the problem data and initial states. The function should simulate the system for the controller gains provided and return matrices X and U which contain state and input trajectories, respectively, obtained from the simulation. D) Use the function wrote in part c) to simulate the system for the solution found in part b). Plot the state and input trajectories to different figures. Note: you can choose initial conditions arbitrarily as Iong as they are not close to zero