CAP $6629$: Reinforcement Learning

Spring $2024$

Course project $2$

Submission: Two files $($one report in $.$pdf and one $.$ipynb$/$code$).$

Please follow the project report guidelines and submit the report with setup, results and analysis.

In project $1,$ you may realize that when you have a large grid world maze setup, it takes a long time for the agent to learn a value table. One way to eliminate this challenge is to use neural networks to approximate the value function. There are two options provided below and you may choose either one to implement.

A$.$ Based on your results in project $1,$ you can choose to build a neural network $($or deep neural network$)$ to approximate your obtained $Q$ or $V$ table.

B$.$ You can design another complex grid world example and develop the QNN $($or deep QNN$)$ method based on that.

Either way, you are using a neural network to generate your $Q$ or $V$ value so that you can guide the agent to move to achieve the goal.

Report requirments:

$**$Any Al$-$generated content is not allowed in the report and$/$or code.

Maze Description: Design your own grid world example and describe it at the beginning of the report.

Problem Formulation: Define your states, actions, and rewards.

Q Network Design: Design and implement your $Q$ network.

Pseudo Code: Provide the pseudo code in the report.

Results and Discussions: Show the convergence process of mean square error $($objective function$)$ and the weights trajectories.

Reference: cite all your reference here.