The answer to this stochastic dynamic programming is attached. Please go through explicit detail as to how each value in the stage table values were calculated. I really want to understand this problem and will give a thumbs up for a good explanation.

6. (20 Points) Queuing problems concern policies on how to serve cust waiting in line for a service. omers Who a Consider the following queueing problem. There is a limit of 1 on queue size e period, there is a probability that any person waiting in line will be served, in which case e queue goes to zer However,there is also a probability of 1/2 that an (thus, the system state is either 0 or 1 person waiting In a given tim addi tional person will arrive in any period. when someone else arrives, then the arrival is turned aay If there is already one person in line There are two options for operating the queue. Option I has a service ty in any period of 3/4 and the transition probability matrix for the process is consequently: To: 0 From 0 5 .5 1.75 .25 and the corresponding reward matrix R (the arc costs for each possible is given by: To: 01 From 0 1 42 negative entries correspond to some waiting costs and the s the reward The other possible option has the following probability and reward matrices: received when a customer is processed by the system To: 0 1 From 0 1/2 1/2 To: 0 1 From 0 0-4 That is, there is a 100% probability of serving any higher cost to operating the queue. customer in queue, but there is a The object is to control the system over 3 time periods in maximize our expected income over these four periods. the assumption that customer left in the system at the end of these 3 periods means a penalty to us of 5 units (by way of compensation for not being served) (This is a finite horizon problem, so use d the time horizon, what is the optimal decision at each state at t=0,1,and 2? such a manner as to ima ynamic programming.) That is, if t-3 is It might help to dr aw a dynamic programming network showing decision nodes and r-hance nodes.) 23