I have uploaded one screenshot $od$ question and one screenshot whicg $is$ similairly solved with this process $.$ I hope this helps $.$ Please solve this question for $meas$ soon $as$ possible

$(30$ Ipoints$)$ Consider a biomanufacturing process that produces biopharmaceuticals $ina$

batch process. A batch can $be$ fermented for a maximum $of4$ days. During that time, the

product $is$ being formed $in$ a non$-$linear fashion with the revenues representing the reward

achieved $if$ the batch $is$ harvested after the respective batch fermentation duration. The

problem $is,$ that the batch can also fail which results $in$ zero revenue. The probability

$of$ a batch failure increases with the batch fermentation duration. Fore example, a batch

which has been fermented for $2$ days has a probability $of10\%to$ fail within the next day $of$

fermentation.

The operating cost $to$ ferment the batch for another day $isC=-\frac{10}{M}U.$ The revenue and

batch failure risk per batch duration are given $in$ Table $1.$ The operations manager wants $to$

determine the optimal harvest day. Note that a batch that has failed $or$ has been fermented

for four days needs $tobe$ harvested. A harvested batch $is$ not replaced with a new one, hence

the problem can $be$ modelled $asan$ indefinite$-$horizon $MDP$ problem. This means that the

dynamic process stops whenever the batch $is$ harvested.

Table $1$: Process parameter for batch biomanufacturing process with batch failure risk.

$(a)(10$ points$)$ Formulate the state space, action space and rewards for each state and

action pair for this dynamic decision$-$making problem.

$(b)(10$ points$)$ Visualize the transition probabilities $in$ a graph with the nodes being the

states and the arcs stating the action and transition probabilities. You can use the

template provided $in$ the appendix. Note the additional "Harvested $(H)"$ state, which

$is$ a terminal $(sink)$ state with $no$ allowed action and $no$ immediate reward.

$(c)(10$ points$)$ Write down the Bellman optimality equation for each state explicitly and

determine the optimal actions for each state. What $is$ the optimal harvest time point

for this batch fermentation? $t=6$:$V_6(1)=80$

$V_6(2)=60$

$V_6(3)=50$

$V_6(6)=5$

$t=5$:$V_5(1,a=1)=79\mathrm{.}8,V_5(2,a=0)=60\mathrm{.}45$

$V_5(5,a=1)=9\mathrm{.}8,V_5(F,a=0)=0\mathrm{.}0$

Peric

$t=4$:$V_4(1)=ma{x}_{\text{a}\text{i}\text{n}\{0,1\}}\{\begin{array}{ccc}p(F|1,& 0)*[r(1,0,F)+V_5(F)]+p(2|1,0)*[r(1,0,& 2)+V_5(2)]=0\mathrm{.}05*[-0\mathrm{.}6+0]+0\mathrm{.}95*[18\mathrm{.}4+60\mathrm{.}45]=74\mathrm{.}88\end{array}$

$r(1,1)+V_5(1)=-0\mathrm{.}2+79\mathrm{.}8={79\mathrm{.}6}^{*}$

$V_4(4)=ma{x}_{\text{a}\text{i}\text{n}\{0,1\}}\{\begin{array}{ccc}p(F|4,& 0)*[r(4,0,F)+V_5(F)]+p(5|4,0)*[r(4,0,& 5)+V_5(5)]=0\mathrm{.}3*[-1\mathrm{.}7+0]+0\mathrm{.}7*[13\mathrm{.}8+9\mathrm{.}8]=16\mathrm{.}01\end{array}$

$r(4,1)+V_5(1)=-50\mathrm{.}2+79\mathrm{.}8={29\mathrm{.}6}^{*}$

$V_4(F)=ma{x}_{\text{a}\text{i}\text{n}\{0,1\}}\{\begin{array}{ccc}p(F|F,& 0)*[r(F,0,& F)+V_5(F)]=1\mathrm{.}0*[0+0]={0\mathrm{.}0}^{*}\end{array}$

$r(F,1)+V_5(1)=-80\mathrm{.}2+79\mathrm{.}8=-0\mathrm{.}4$

$t=3$:$V_3(3)=ma{x}_{\text{a}\text{i}\text{n}\{0,1\}}\{\begin{array}{ccc}p(F|3,& 0)*[r(3,0,F)+V_4(F)]+p(4|3,0)*[r(3,0,& 4)+V_4(4)]=0\mathrm{.}2*[-1\mathrm{.}5+0]+0\mathrm{.}8*[15\mathrm{.}7+29\mathrm{.}6]=35\mathrm{.}94\end{array}$

$r(3,1)+V_4(1)=-30\mathrm{.}2+79\mathrm{.}6={49\mathrm{.}4}^{*}$

Production and Supply Chain Management $|$ TUM School $of$ Management $|$ Technical University $of$ Munich

$10$