The objective of this project is to use the perceptron learning method and design an artificial neural network (ANN) to train a simple system (single layer perceptron) for the recognition of handwritten digits (0, 1, , 9). Design a fully connected network structure of 784 input nodes and 10 output nodes. The input to your single layer network architecture will be a set of binary pixels representing a 2828 image of handwritten digits. The output should indicate which of the digits (0,....,9) is in the input image.

Use the MNIST database of handwritten digits available on Blackboard-Homepage-Handwritten Digits Dataset. Select a subset of the MNIST database consisting around 500 images of handwritten digits (0,...,9) for training the system, and use another 100 images for testing the system. Create binary or bipolar images of handwritten digits from gray scale images available in MNIST by simple thresholding (indicate the threshold value you used). Plot a learning curve that illustrates the mean square error versus iterations. (One iteration: apply all the training inputs once to the network and compute the mean square error). Plot the percentage error in testing your handwritten digit recognition system as a bar chart. (Mean error occurred while testing each digit with the test data). Task #1: Repeat this experiment for different learning rate parameters (at least 3 experiments. Start with a large value and gradually decrease to a small value). Task #2: Repeat Task #1 with a large database. The first 10000 images for training (image indexes from 0-10000) and test with another 1000 images (image indexes from 20000-21000). Task #3: Repeat Task #2 with multilevel data (without thresholding the input data, normalize the input data, use sigmoid function for output thresholding). What can you note comparing with part 2? Task #4: Compare your results with the SVM results (what you have got from the last project).