Part $2.$ Adding CNN and fully connected layers to recognize handwritten digits on PyTorch $[40$ points$]$

In this part, you will deal with the MNIST Database. The MNIST Database is a collection of samples of handwritten digits from many people, originally collected by the National Institute of Standards and Technology $($NIST$),$ and modified to be more easily analyzed computationally. We will use a tutorial and sample software provided:

$$ Read and run a tutorial to be familiar with how to add CNN layers into PyTorch.

$$ Download hw$4-$part$2.$ipynb and run it to be familiar with the code. Currently it contains two fully connected

layers with softmax.

$$

$$ Then, experiment with at least $3$ alternative network topologies and hyper$-$parameters $($e$.$g$.,$ different # of CNN$/$fully$-$connected layers, # of epochs, # of hidden units, learning rate, batch size, and different activation functions$).$

$$ Save and summarize the results and report them.

$$ Through the experiment, what is the best configuration? What prediction accuracy on the test set you got?

What did you learn?$$Refer to the detailed instruction about the CNN layer. The main difference between the tutorial and this given hw$4\_$part.ipynb is the input image of hw$4\_$part.ipynb has only $1$ channel $($i$.$e$.,$ gray scale$).$ Report results of two fully connected layers without CNN and with CNN$.$

Your job is to add one CNN layer with one pooling layer before the two fully connected layers with softmax.