$3$ CNNs vs Dense Layers

Problem $8$: Consider again the model you built in Problem $4.$ This was a relatively simple model with

vanilla dense layers. Consider constructing a simple CNN model in the following way:

Pass the input image $(28281)$ into a convolutional layer and an activation function.

Flatten the result.

Pass the flat result into a number of dense layers $($and activation functions$).$

Pass the result through a softmax layer to get class probabilities.

With a single convolutional layer $($kernel size and number at your discretion$),$ find the smallest model you can

$($in terms of total number of parameters$)$ that ultimately matches or exceeds the performance of the model

you found in Problem $4.$ How did you go about your neural architecture search to answer the question?

For the CNN architecture you find and the original architecture from Problem $4,$ plot the training and testing

loss over training time for comparable batch sizes, step sizes, and optimizer $($be clear about the choices you

are making$).$

Problem $9$: Consider Problem $8,$ but you are allowed two stacked convolutional layers $($of different kernel

sizes $/$ numbers$).$ Can you beat the network from Problem $8,$ in terms of performance vs parameter count?

Bonus: For the dense model from Problem $4,$ and the model from Problem $9,$ find instances where the models

fail $($incorrectly classifying the image$).$ Are the mistakes being made reasonable, to your eye? Are the models

making different kinds of mistakes?