$1.$ Load the image dataset: load the Digits dataset from a predefined path in the MATLAB toolbox. The imageDatastore function handles the loading of image data. All subfolders are included and folder names are used as labels. The images are resized to $32$x$32$ pixels for compatibility with the LeNet$-5$ architecture. The data is then split into training and validation sets, with $70\%$ used for training and $30\%$ for validation.

$\%$ Load the Digits dataset

digitDatasetPath $=$ fullfile$($toolboxdir$(\text{'}$nnet$\text{'}),$ 'nndemos', $...$

'nndatasets', 'DigitDataset'$)$;

imds $=$ imageDatastore$($digitDatasetPath$,...$

'IncludeSubfolders', true, 'LabelSource', 'foldernames'$)$;

imds.ReadFcn $=$ @$($loc$)$imresize$($imread$($loc$),[32,32])$;

$\%$ Split the data into training and validation datasets

$[$imdsTrain$,$ imdsValidation$]=$ splitEachLabel$($imds$,0\mathrm{.}7,$ 'randomized'$)$;

Define the architecture of the CNN using the layers function: It begins with an input layer expecting images of size $28$x$28$ with one channel $($greyscale$).$ Following this are alternating convolutional, ReLU activation, and max pooling layers. Then, there are several fully connected layers with ReLU activation, ending with a softmax layer and a classification output layer. Refer to the MATLAB documentation to understand how to define each of these layers. In Matlab, the loss function is implicitly defined through the classificationLayer. Refer to the MATLAB documentation to understand how to define each of these layers.

$\%$ Define the LeNet$-5$ architecture

layers $=[$

imageInputLayer$([32321],$'Name','input'$)$

convolution$2$dLayer$(5,6,$'Padding','same','Name','conv$\_1\text{'})$

averagePooling$2$dLayer$(2,$'Stride',$2,$'Name','avgpool$\_1\text{'})$

convolution$2$dLayer$(5,16,$'Padding','same','Name','conv$\_2\text{'})$

averagePooling$2$dLayer$(2,$'Stride',$2,$'Name','avgpool$\_2\text{'})$

fullyConnectedLayer$(120,$'Name',$\text{'}$fc$\_1\text{'})$

fullyConnectedLayer$(84,$'Name',$\text{'}$fc$\_2\text{'})$

fullyConnectedLayer$(10,$'Name',$\text{'}$fc$\_3\text{'})$

softmaxLayer$(\text{'}$Name$\text{'},$'softmax'$)$

classificationLayer$(\text{'}$Name$\text{'},$'output'$)]$;

$2.$ Configure training options using the trainingOptions function. Set parameters like the optimisation algorithm, learning rate, batch size, and number of epochs.

In this block, you specify the training options. You use stochastic gradient descent with momentum $($sgdm$)$ as your optimization algorithm, set an initial learning rate of $0\mathrm{.}0001,$ and limit the training to a maximum of $10$ epochs. The training data will be shuffled at the start of each epoch. The progress of training will be displayed as a plot.

$\%$ Specify the training options

options $=$ trainingOptions$(\text{'}$sgdm$\text{'},...$

'InitialLearnRate',$0\mathrm{.}0001,...$

'MaxEpochs',$10,...$

'Shuffle','every$-$epoch', $...$

'ValidationData',imdsValidation, $...$

'ValidationFrequency',$30,...$

'Verbose',false, $...$

'Plots','training$-$progress'$)$;

$3.$ Train the CNN using the trainNetwork function, passing in the image data, the layer definitions, and the training options. Please keep in mind that it may require some time to train your model.

$\%$ Train the network

net $=$ trainNetwork$($imdsTrain$,$layers,options$)$;

CNN Performance Evaluation

$4.$ After the network is trained, it's used to classify the images in the validation dataset. The classify function is used to perform this classification. The accuracy of the network on the validation images is then calculated and printed. This is the proportion of images that were correctly classified by the network. It's calculated as the number of correctly classified images divided by the total number of images in the validation set.

$\%$ Classify validation images and compute accuracy

YPred $=$ classify$($net$,$imdsValidation$)$;

YValidation $=$ imdsValidation.Labels;

accuracy $=$ sum$($YPred $==$ YValidation$)/$numel$($YValidation$)$;

fprintf$(\text{'}$Accuracy of the network on the validation images: $\%$f

$\text{'},$ accuracy$)$;

VI$.$ Task $3$: Improving the Performance of the CNN Algorithm

Guidance on several methods to improve the image classification accuracy:

$1.$ Regularisation Techniques: In your quest to enhance the CNN accuracy, a key strategy involves regularisation methods such as L$1$ and L$2$ norm regularisation. For understanding of L$1$ and L$2$ norm regularisation, please refer to $[$Tewari$,2021]$ and $[$Bilogour et al$,2023]$ from the Kaggle tutorial. The L$1$ and L$2$ norm regularisation methods are instrumental in overcoming overfitting. You can implement L$1$ and L$2$ regularisation in MATLAB by adjusting the WeightL$1$Factor and WeightL$2$Factor properties within the fullyConnectedLayer:

fullyConnectedLayer$(120,$'Name',$\text{'}$fc$\_1\text{'},$ 'WeightL$1$Factor', $0\mathrm{.}001,$ 'WeightL$2$Factor', $0\mathrm{.}001)$

Additionally, dropout is another valuable regularisation method for reducing overfitting in neural networks. It can be integrated using the dropoutLayer function, further helping in the model's robustness. For your report present results with two regularisation methods, e$.$g$.$ with L$1$ or L$2$ norm regularisation or dropout or combinations of them.

$2.$ Activation Functions: Another way to improve the CNN accuracy and efficiency is with different activation functions such as ReLU, Sigmoid, or Tanh. These functions play a pivotal role in enabling neural networks to capture complex data patterns and nonlinear relationships. This can be achieved by integrating functions such as reluLayer for ReLU activation. For your report present results with two activation functions.

$3.$ CNN Architecture Exploration: Delving into various CNN architectures is a critical step towards optimising your networks performance. You can explore modifications such as altering the number of convolutional or fully connected layers, or experimenting with different types of pooling operations, such as max pooling and average pooling. For a more advanced exploration, you might consider studying complex architectures such as AlexNet $($Krizhevsky et al$.,2012).$ Please keep in mind that increasing the models complexity may reduce computational efficiency.

The function dropoutLayer defines a dropout layer to your model.

$4.$ Hyperparameter Tuning: The fine$-$tuning of hyperparameters, including the learning rate and the number of epochs, is an essential aspect of enhancing CNN performance. Adjusting these parameters can lead to significant improvements in training efficacy and model accuracy. However, it is crucial to be aware of the potential for overfitting, particularly when increasing the number of epochs. To counteract this effect, you might need to incorporate regularisation methods, which can help in maintaining a model that generalises well to new data. The process of hyperparameter tuning is iterative and requires careful observation and analysis to identify the optimal configuration for your specific model and dataset. For your report present results with two sets of hyperparameters those that give the best results and another set that provides less accuracy and efficiency.

The methods outlined above are recommended and could also be combined for potentially enhanced outcomes. Furthermore, you are encouraged to experiment with other innovative approaches to refine your CNN$\text{'}$s accuracy. It is essential to conduct a thorough analysis of your results, delving into the implications and effectiveness of each method employed, to gain deeper insights into your model's performance.

$1.$ Provide your results of the accuracy and the analysis in your report.

$2.$ Calculate the Precision, Recall, and the F$1$ score of your classification results.

$3.$ Can you improve the results? Please explain how you improve the accuracy and analyse the results in detail.