Finding the Cost-Minimizing Threshold for your Model

Now that you have, hopefully, developed your model to the point where it is relatively robust across the training set and test set, your boss at the bank finally gives you its current rough estimate of the banks average costs for each type of classification error.

[Note that all bank models here include only profits and losses within three years of when a card is issued, so the impact of out-years (years beyond 3) can be ignored.]

Cost Per False Negative: $5000

Cost Per False Positive: $2500

For the 600 individuals that were automatically given cards without being classified, the total cost of the experiment turned out to be 25%*($5000)*600 or $750,000. This is $1,250 per event.

Only models with lower cost per event than $1,250 should have any value.

Question: What is the threshold score on the Training Set data for your model that minimizes Cost per Event? You will need this number to answer later questions.