USE THIS MODEL TO ANSWER THE UPCOMING QUESTIONS

MODEL:

import numpy as np

import pandas as pd

import tensorflow as tf

from tensorflow.keras import layers, models

from sklearn.metrics import confusion$\_$matrix

import seaborn as sns

import matplotlib.pyplot as plt

data:

x$\_$test, x$\_$train, x$\_$val, y$\_$test, y$\_$test$\_$num, y$\_$train, y$\_$train$\_$num, y$\_$val, y$\_$val$\_$num

tf$.$keras.utils.set$\_$random$\_$seed$(42)$

tf$.$config.experimental.enable$\_$op$\_$determinism$()$

num$\_$classes $=4$

model $=$ models.Sequential$([$

layers.Conv$2$D$(4,(5,5),$ activation$=$'relu', padding$=$'same', input$\_$shape$=(80,80,1)),$

layers.MaxPooling$2$D$((2,2),$ strides$=(2,2)),$

layers.Conv$2$D$(8,(5,5),$ activation$=$'relu', padding$=$'same'$),$

layers.MaxPooling$2$D$((2,2),$ strides$=(2,2)),$

layers.Conv$2$D$(16,(5,5),$ activation$=$'relu', padding$=$'same'$),$

layers.MaxPooling$2$D$((2,2)),$

layers.Flatten$(),$

layers.Dense$(50,$ activation$=$'relu', kernel$\_$regularizer$=$tf$.$keras.regularizers.l$2(0\mathrm{.}01)),$

layers.Dense$($num$\_$classes, activation$=$'softmax'$)$

$])$

model.compile$($loss$=\text{'}$ca

tegorical$\_$crossentropy', optimizer$=$ 'adam', metrics$=[\text{'}$accuracy$\text{'}])$

history $=$ model.fit$($x$\_$train, y$\_$train, epochs$=20,$ batch$\_$size$=32,$ validation$\_$data$=($x$\_$val, y$\_$val$),$ verbose$=1)$

# Plotting training & validation accuracy values$/$loss values

plt$.$plot$($history$.$history$[\text{'}$loss$\text{'}],$ label$=$'Training Loss'$)$

plt$.$plot$($history$.$history$[\text{'}$val$\_$loss'$],$ label$=$'Validation Loss'$)$

plt$.$xlabel$(\text{'}$Epochs$\text{'})$

plt$.$ylabel$(\text{'}$Loss$\text{'})$

plt$.$legend$()$

plt$.$show$()$

plt$.$plot$($history$.$history$[\text{'}$accuracy$\text{'}],$ label$=$'Training Accuracy'$)$

plt$.$plot$($history$.$history$[\text{'}$val$\_$accuracy'$],$ label$=$'Validation Accuracy'$)$

plt$.$xlabel$(\text{'}$Epochs$\text{'})$

plt$.$ylabel$(\text{'}$Accuracy$\text{'})$

plt$.$legend$()$

plt$.$show$()$

# Evaluating the model on the test set

test$\_$loss, test$\_$acc $=$ model.evaluate$($x$\_$test, y$\_$test$)$

print$(\text{'}$Test Loss:$\text{'},$ test$\_$loss$)$

print$(\text{'}$Test Accuracy:$\text{'},$ test$\_$acc$)$

# Predictions

train$\_$predictions $=$ model.predict$($x$\_$train$)$

train$\_$pred$\_$labels $=$ np$.$argmax$($train$\_$predictions, axis$=1)$

train$\_$true$\_$labels $=$ np$.$argmax$($y$\_$train, axis$=1)$

train$\_$conf$\_$matrix $=$ confusion$\_$matrix$($train$\_$true$\_$labels, train$\_$pred$\_$labels$)$

sns$.$heatmap$($train$\_$conf$\_$matrix, annot$=$True, fmt$=\text{'}$d$\text{'})$

plt$.$title$(\text{'}$Confusion Matrix $-$ Training Set'$)$

plt$.$show$()$

test$\_$predictions $=$ model.predict$($x$\_$test$)$

test$\_$pred$\_$labels $=$ np$.$argmax$($test$\_$predictions, axis$=1)$

test$\_$true$\_$labels $=$ np$.$argmax$($y$\_$test, axis$=1)$

test$\_$conf$\_$matrix $=$ confusion$\_$matrix$($test$\_$true$\_$labels, test$\_$pred$\_$labels$)$

sns$.$heatmap$($test$\_$conf$\_$matrix, annot$=$True, fmt$=\text{'}$d$\text{'})$

plt$.$title$(\text{'}$Confusion Matrix $-$ Test Set'$)$

plt$.$show$()$

Answer to the following questions:

How many parameters does the model train? Before performing the training, do you expect this model to overfit? Which aspects would influence the overfitting $($or not$)$ of this model?

Plot the loss function and the accuracy per epoch for the train and validation sets.

Which accuracy do you obtain on the test set?

Using the function plot$\_$confusion$\_$matrix plot the confusion matrices of the classification task on the train set and test set. What do you observe from this metric? Which classes display more correct predictions? And wrong?

Using the function ind$\_$correct$\_$uncorrect extract the indexes of the training data that were predicted correctly and incorrectly, per each class. For each music genre, perform the following steps:

Using the function plot$\_$spectrograms plot the $12$ mel spectrograms of the first $6$ data points which were predicted correctly and the first $6$ which were predicted wrongly. Do you observe some differences among music genres?

Using the function print$\_$wrong$\_$prediction print the predicted classes of the first $6$ data points which were predicted wrongly.

Using the Grad$-$CAM method, implemented in the function plot$\_$gradcam$\_$spectrogram, print the heatmaps of the last pooling layer for the same $12$ extracts $(6$ correct $+6$ wrong$).$ Comment on the heatmaps obtained. Do you observe differences among the heatmaps of different music genres? Can you understand why the model got some predictions wrong?

Comment on the previous question: what are your thoughts about the applicability of the Grad$-$CAM tool on these data?

P$2-$ Disentangling time and frequency

The images we are using in this assignment are different from a usual picture: the x and y axes carry different meanings. With the tools we are exploring during lectures and seminars, can you propose a CNN architecture that takes into account differently the time and frequency components of the spectrograms?

Present and describe the architecture you have chosen and justify the rationale behind it$.$ Plot training and validation loss and accuracy over $20$ epochs $($this time you can use the GPU runtime if the model is slow to train$).$ Print the accuracy on the test set and the confusion matrices on the training and test sets.