Elite Air (EA)2 is a business-class only airline advertising complete meals for all its passengers.

EA must choose, then update the number to order qe, at epochs e = 4,

c, 0, which is initially set to the number of booked and standby passengers b4 known at that time. Values of both qe and be can range between 0, 1,

c, B!the seating capacity of the flight. As flight time approaches, meal requirements and passenger loads are reevaluated at every 4-hour epoch.

The predicted number of passengers who must be served varies stochastically over the epochs as bookings are clarified, but values are independent of meal planning decisions, specifically p[be, be - 1]!the probability that estimate be is updated to be - 1 during epoch e = 4,

c, 1.

Order quantities qe are also reviewed at each e.

In epochs e = 4 and 3, meals can be ordered at standard cost c dollars each. Thereafter, extra meals may be added at late-order cost $1.8c. Meals already on order may be cancelled, but the savings will only be $0.4c, not the full original amount.

EA wants to develop a meal ordering policy that assures q0 = b0 at departure time, while minimizing total expected cost of meal purchase and return.

(a) Formulate EA’s challenge as a MDP with multiple states and stages, including identifying all the elements of definition 9.44 .

Use bivariate states 1qe, be2 with qe! the current number of meals ordered, and be!

the corresponding estimate of passengers expected. Round any fractional value arising in computations to the nearest integer.

(b) Sketch a digraph depicting your model of part

(a) over nodes for the states and stages, plus arcs for possible transitions.

You need not insert all parameter details, but do show exemplars of all transition arcs with the decision to which they are attached, the reward they would realize, and the associated probability.

(c) Form the functional equations over all states and stages for your model of (a).