4 TensorFlow/ Keras tutorial Use the TensorFlow {3: Keras) tutorial Basic classication: Classify ges of clothing to learn how to use Keras to create and train a neural network model (in particular, a Multi-layer Feed- Forward network in this exercise). The task here is Fashion W151", a "drop-in replacement" for MNIST. That is, it's got the same structure, but different types of images. 4.1 Preliminary notes: - You can run the code directly on Google IColab by going to the Basic classication tutorial link above, then clicking the Run in Google Colab button near the top. That will open up a copy of the notebook in Colab which you can modify. Alternatively, you can download the notebook (with the Download notebook button), and run it on a dierent cloud platform or on your own machine, but then you'll have to ensure that the version of tensorow is consistent with what's in the notebook. (It uses TF 2.3.1, but it might work on other versions of TF 2.) If you prefer running locally on your machine, you need to install TensorFlow and Keras by yourself and congure your system. +3 Running the code: If you can run your code with hardware acceleration (GPU or TPU), it will greatly increase the speed.Ir With Colab, you can do this with Runtime > Change Runtime Type > Hardware Accelerator > GPU or TPU. There is a limit on how long you can use these, and it's not a published formula, but it will make your training much faster. You might want to disconnect your session in between training runs to avoid hitting the limit. You can also sign up for "Colab Pro" (under settings). I think it's just $10fmonth. It provides faster hardware and more relaxed runtime restrictions. - You may modify any line in the tutorial code to do experiments. However the following must NOT be changed (otherwise your code won't work at all or work well): a In the model, the first layer has to be Flatten with input_shepe=(28, 28), and the last layer has to be Dense with 10 nodes). o In compile( ), the loss function has to be sparse_categorical_crossentropy as shown below because the target (y) in Fashion MNIST data is an integer (0 through 9 in this case) instead of a binarized "one-hot" vector. sparse_categorical_crossentropy (from_logits=True) . To look up the descriptions of Keras hyperparameters, look at Keras Documentation, for example, Guide on the Sequential model and the API on models, layers, etc. 4.2 Objectives The goal of this exercise is twofold: A. To run various model configurations and hyperparameter values to empirically examine their effects in learning; and B. To experience searching for an optimal model/hyperparameter setting for neural networks (i.e., a journey to tune the model). Read the NNDL Chapter 3, section How to choose a neural network's hyper-parameters as a guide. 4.3 Experiment Description: . Experiment with at least four of the following parameters: o Number of nodes in hidden layers o Number of hidden layers o Activation function for hidden layers o Learning rate (eta; set through Keras Optimizer) o Mini-batch size (batch_size in fit()) o Regularization methods (set through Keras Regularizer) o Dropout (using Keras Corelayer)1. rd 1 l. at. U different values while xing the values for all other parameters. After each experiment, choose the value of the parameter that gave the best performance, and carry it over to the next experiment. Note that "bestporformonoe" can be dened from various perspectives, e. g., classication accuracy (with respect to training or test set?), loss value (and with respect to train or test set?), speed of learning, final accuracyoss after a fixed number of epochs (again with respect to training or test set?), and degree of overtting. Think about it carefully for each parameter before deciding the value to carry over to the next experiment. Also, think about Whether the metric is comparable between one hyperparameter and another. Alternatively you can do a systematic search for optimal parameters, for instance by using grid search or other methods. But you should produce results that provide comparisons between the different parameter settings. {To her?) your foam!) Write at least 300 words on your experiments and results, including your comments on the value you chose as the best one for each parameter. Include this in the REPORT pdf described below. +3 Write in detail your expectations for the various values of those parameters before the experiment, and your reactions on the results {as you had expected or otherwise, and possible reasons why). Include the results in your report (not in the notebook alone) in a chart or table for each parameter (REQUIRED). For comparison, you may want to look at another Keras tutorial which uses a validation set and plots the behavior of training (training vs. validation)