Part $4$: Evaluation

$4\mathrm{.}1$ Evaluating your siamese network

In this section you will learn how to evaluate a Siamese network. You will start by loading a pretrained model, and then you will use it to predict. For the prediction you will need to take the output of your model and compute the cosine loss between each pair of questions.

$4\mathrm{.}2$ Classify

To determine the accuracy of the model, you will use the test set that was configured earlier. While in training you used only positive examples, the test data, Q$1\_$test, Q$2\_$test and y$\_$test, is set up as pairs of questions, some of which are duplicates and some are not. This routine will run all the test question pairs through the model, compute the cosine similarity of each pair, threshold it and compare the result to y$\_$test $-$ the correct response from the data set. The results are accumulated to produce an accuracy; the confusion matrix is also computed to have a better understanding of the errors.

Exercise $04$

Instructions

Loop through the incoming data in batch$\_$size chunks, you will again define a tensorflow.data.Dataset to do so$.$ This time you don't need the labels, so you can just replace them by None,

compute v$1,$ v$2$ using the model output,

for each element of the batch $-$ compute the cosine similarity of each pair of entries, v$1[$j$],$v$2[$j$]-$ determine if d $>$ threshold $-$ increment accuracy if that result matches the expected results $($y$\_$test$[$j$])$ Instead of running a for loop, you will vectorize all these operations to make things more efficient,

compute the final accuracy and confusion matrix and return. For the confusion matrix you can use the tf$.$math.confusion$\_$matrix function.

# GRADED FUNCTION: classify

def classify$($test$\_$Q$1,$ test$\_$Q$2,$ y$\_$test, threshold, model, batch$\_$size$=64,$ verbose$=$True$)$:

$"""$Function to test the accuracy of the model.

Args:

test$\_$Q$1($numpy$.$ndarray$)$: Array of Q$1$ questions. Each element of the array would be a string.

test$\_$Q$2($numpy$.$ndarray$)$: Array of Q$2$ questions. Each element of the array would be a string.

y$\_$test $($numpy$.$ndarray$)$: Array of actual target.

threshold $($float$)$: Desired threshold

model $($tensorflow$.$Keras.Model$)$: The Siamese model.

batch$\_$size $($int$,$ optional$)$: Size of the batches. Defaults to $64.$

Returns:

float: Accuracy of the model

numpy.array: confusion matrix

$"""$

y$\_$pred $=[]$

test$\_$gen $=$ tf$.$data.Dataset.from$\_$tensor$\_$slices$((($test$\_$Q$1,$ test$\_$Q$2),$None$)).$batch$($batch$\_$size$=$batch$\_$size$)$

### START CODE HERE ###

pred $=$ None

$\_,$ n$\_$feat $=$ None

v$1=$ None

v$2=$ None

# Compute the cosine similarity. Using $`$tf$.$math.reduce$\_$sum$`.$

# Don't forget to use the appropriate axis argument.

d $=$ None

# Check if d$>$threshold to make predictions

y$\_$pred $=$ tf$.$cast$($None$,$ tf$.$float$64)$

# take the average of correct predictions to get the accuracy

accuracy $=$ None

# compute the confusion matrix using $`$tf$.$math.confusion$\_$matrix$`$

cm $=$ None

### END CODE HERE ###

return accuracy, cm