Consider the following environment of PacMan

For the environment design a Reinforcement Learning Agent (Pacman), the objective of the agent is to figure out the best actions the agent can take at any given state. The rules of the game are as follows:

Every move has a reward of -1

Consuming a food pellet will have a reward of +10

If pacman collides with a ghost, then the reward will be -500

If the pacman has eaten all the food pellets without colliding with the ghosts, then the reward will be +500

Assume a discount factor of 0.8

The action noise is 0.3 (the consequences are the same as in the grid world example)

The environment is static i.e. no ghosts are moving

The actions for pacman are Up, Down, North and Right

You can cross the walls Use Q-Learning to figure out the best action at every state. Show your working for every iteration of Q-Learning.

6,4 0,0 6,4 0,0