Consider the equipment location example; its text is repeated for convenience. A service man moves between 4 sites, with site 1 denoting home office, and 2, 3, and 4 denoting remote sites. Work at sites 2,3, and 4 requires the use of an equipment trailer. The cost of relocating the equipment trailer between the sites is d(), k) = 300 for k + j. The cost ck,j) of using the trailer is 100 if the work is at site k >1 and trailer is at site j + k with j > 1; 50 if j = k and j > 1, and 200 if the work is at k > 1 and the trailer is at j = 1 (home office). If the service man is at the home office, no work is done and cost is zero. At any time, the service man knows their location, observes the location of the next job (or 1 if no job is to be done) and decides whether and where to move the trailer. The transition matrix between job locations is P= [0.1 0.3 0.3 0.37 0.0 0.5 0.5 0.0 0.0 0.0 0.8 0.2 [0.4 0.0 0.0 0.6 For example, if the current location (job) is 2, the probability that the next job is at 3 is 0.5. Assume the discount factor of 0.95. (a) Formulate a Markov decision problem to help the repairment decide on the movement of the trailer: define the state and control space and write the dynamic programming equations. (b) Solve these equations by the value iteration method (starting from 0) and calculate lower and upper bound at each step. (c) Solve the problem by the policy iteration method starting from "do not move trailer from base at each state. (d) Solve the problem by linear programming. Consider the equipment location example; its text is repeated for convenience. A service man moves between 4 sites, with site 1 denoting home office, and 2, 3, and 4 denoting remote sites. Work at sites 2,3, and 4 requires the use of an equipment trailer. The cost of relocating the equipment trailer between the sites is d(), k) = 300 for k + j. The cost ck,j) of using the trailer is 100 if the work is at site k >1 and trailer is at site j + k with j > 1; 50 if j = k and j > 1, and 200 if the work is at k > 1 and the trailer is at j = 1 (home office). If the service man is at the home office, no work is done and cost is zero. At any time, the service man knows their location, observes the location of the next job (or 1 if no job is to be done) and decides whether and where to move the trailer. The transition matrix between job locations is P= [0.1 0.3 0.3 0.37 0.0 0.5 0.5 0.0 0.0 0.0 0.8 0.2 [0.4 0.0 0.0 0.6 For example, if the current location (job) is 2, the probability that the next job is at 3 is 0.5. Assume the discount factor of 0.95. (a) Formulate a Markov decision problem to help the repairment decide on the movement of the trailer: define the state and control space and write the dynamic programming equations. (b) Solve these equations by the value iteration method (starting from 0) and calculate lower and upper bound at each step. (c) Solve the problem by the policy iteration method starting from "do not move trailer from base at each state. (d) Solve the problem by linear programming