Consider the famous Snake game. The player here is an autonomous

agent. The player controls the game with four controls $-$ up$,$ down,left,

right. There will be food anywhere on the screen.The food disappears in

$10$ secs$.$ The player has to control the snake in a way it eats food and he

has to avoid the frame$$s borders otherwise the snake dies.Each time the

snake eats food, the body length of the snake is increased. During the

game the player has also to avoid the snake$$s body otherwise the game

terminates, as well. The rewards are Eat food $=+10,$ Game over $=-10,$

else $=0.$ Make the required assumptions and state them clearly.Consider the famous Snake game. The player here is an autonomous

agent. The player controls the game with four controls $-$ up$,$ down,left,

right. There will be food anywhere on the screen.The food disappears in

$10$ secs$.$ The player has to control the snake in a way it eats food and he

has to avoid the frame$$s borders otherwise the snake dies.Each time the

snake eats food, the body length of the snake is increased. During the

game the player has also to avoid the snake$$s body otherwise the game

terminates, as well. The rewards are Eat food $=+10,$ Game over $=-10,$

else $=0.$ Make the required assumptions and state them clearly. $($i$)$ Write down an MDP formulation for the given scenario with an explanation for all design

choices. $[3$ M$]$

$($ii$)$ After some time you find the game not progressing with the rewards. What can go wrong?

$[1$ M$]$

$($iii$)$ Your initial reward is $0$ followed by infinite $10$s$.$ What is the initial and the next expected

return $?$ Let the discount rate be $0\mathrm{.}4[2$ M$]$

$($iv$)$ Suppose you treated this as an episodic task but also used discounting, with all rewards

zero except for $1$ upon failure. What then would the return be at each time? How does this

return differ from that in the discounted, continuing formulation of this task?