Hyperparameter Selection on MNIST

The goal of this assignment is to design, train, and optimize a neural network using PyTorch with only linear layers and non$-$linear activations to achieve at least $\backslash (95\backslash \%\backslash )$ accuracy on the MNIST dataset. You will perform basic hyperparameter tuning to meet this target.

Data preparation

Utilize torchvision.datasets.MNIST to load the training and test datasets. You will need a validation set to optimize your parameters. Use $\backslash (20\backslash \%\backslash )$ of the training data as your validation set.

$1.$ Use torch, torch.nn$,$ torch.optim, torchvision, and other required modules.

$2.$ Load the MNIST dataset.

$3.$ Utilize torchvision.datasets.MNIST to load the training and test datasets.

$4.$ Normalize the images to a range of $\backslash ([0,1]\backslash )$ using torchvision.transforms.

$5.$ Use the DataLoader class for batch processing.

Neural network

Design a neural network architecture using only linear layers. Ensure the model contains the following,

$1.$ An input layer compatible with the MNIST image size.

$2.$ One or more hidden layers.

$3.$ An output layer compatible for multi$-$class classification task and the MNIST dataset.

$4.$ Use appropriate activation and loss functions and explain why they were chosen. Training $\backslash$& Evaluation

Choose your optimizer, learning rate scheduler and implement a training loop to train your model. Keep track of your training and validation losses, you will be asked to plot and discuss them later on$.$ In this stage, you will not be touching the test set in any way. You may only work on training and validation sets. Try to get an accuracy score of at least $\backslash (95\backslash \%\backslash )$ on the validation set. Experiment with different hyperparameters, including:

$-$ Learning rate.

$-$ Number of hidden layers and neurons in each layer.

$-$ Batch size.

$-$ Number of epochs.

Record the hyperparameters and their effects on the model's performance. Calculate the training and validation accuracy for each hyperparameter combination and plot the losses. Report the accuracies in a printed DataFrame for each hyperparameter. Comment on your results and explain the table. Finally, select the best model and calculate the accuracy on the test set. Do not perform any hyperparameter tuning after calculating the test set accuracy.