Objective:

The objective of this assignment is to familiarize you with predictive probabilities in the context of fraud

detection and demonstrate how the choice of cutoff threshold impacts the classification of data.

Data Overview

Below is a table containing two columns: "Transaction Type" $(0$ for legitimate, $1$ for fraudulent$)$ and

"Predicted Probabilities" obtained from a machine learning model. This data represents transactions

that need to be classified as either fraudulent or legitimate based on these predicted probabilities.

Transaction Type Predicted Probabilities

$1[0\mathrm{.}477,0\mathrm{.}523]$

$1[0\mathrm{.}401,0\mathrm{.}599]$

$1[0\mathrm{.}366,0\mathrm{.}634]$

$0[0\mathrm{.}246,0\mathrm{.}754]$

$1[0\mathrm{.}322,0\mathrm{.}678]$

$0[0\mathrm{.}788,0\mathrm{.}212]$

$0[0\mathrm{.}368,0\mathrm{.}632]$

$1[0\mathrm{.}333,0\mathrm{.}667]$

$0[0\mathrm{.}679,0\mathrm{.}321]$

$1[0\mathrm{.}411,0\mathrm{.}589]$

$1[0\mathrm{.}244,0\mathrm{.}756]$

$0[0\mathrm{.}569,0\mathrm{.}431]$

$1[0\mathrm{.}211,0\mathrm{.}789]$

$0[0\mathrm{.}735,0\mathrm{.}265]$

$0[0\mathrm{.}424,0\mathrm{.}576]$

$1[0\mathrm{.}078,0\mathrm{.}922]$

$0[0\mathrm{.}622,0\mathrm{.}378]$

$0[0\mathrm{.}451,0\mathrm{.}549]$

$1[0\mathrm{.}259,0\mathrm{.}741]$

$0[0\mathrm{.}522,0\mathrm{.}478]$

Step $3$: Applying Different Cutoffs

$$ Now it$$s your turn to classify the prediction probabilities into classes $0($legitimate$)$ and $1$

$($fraudulent$)$ using the cutoff threshold $0\mathrm{.}5$ based on the approach illustrated in Step $2.$

$$ Repeat the same procedure with cutoff thresholds $0\mathrm{.}6,0\mathrm{.}7.$

Step $4$: Calculate Confusion Matrix

$$ Calculate the confusion matrix for each of three classifications you made in previous step $($based on

cutoff thresholds $0\mathrm{.}5,0\mathrm{.}6,0\mathrm{.}7).$

$$ Calculate accuracy, precision, recall and f$1$ score for each one.

$$ What is the effect of cutoff threshold based on above observations?

Step $5$: Calculate Cost and Identify Best Classifier

Now consider the following cost function that quantifies the cost associated with making different types

of classification errors $($e$.$g$.,$ false positives and false negatives$)$:

Predicted Class $0$ Predicted Class $1$

Actual Class $00$ $ $80$

Actual Class $1$ $ $500$

$$ Calculate the total cost for each of the classifiers and finally decide which cutoff threshold would be

the best choice?