Assume the same information for Spruce Bank as in Problem 11-39. David believes that within the pool of borrowers he invests in, the timing of when borrowers ultimately default could be different from what he has assumed. That means he might lose more or less than the 65% he has assumed in the payoff matrix. He decides to model “worst case” and “best case” scenarios of losing 75% and 55%, respectively, of the amount of the loan in the event of a default.

Data From Problem 11-39

David Porter is the management accountant at Spruce Bank, where the data science department is leading an initiative to predict whether loans will default or repay. The default rate in the training set is 15%. After building a model on the training set that predicts whether a loan will default or repay, the data scientist applies it to the validation set of 400 observations to evaluate its performance.

2. Assume that Spruce Bank has $1,000 to invest in each loan of the validation sample. If Spruce Bank does not invest in a loan, it keeps the money in a risk-free investment at 3% a year for 3 years (ignore the time value of money). If Spruce invests in a loan that eventually repays, it receives 10% a year for 3 years. If Spruce invests in a loan that eventually defaults, Spruce loses 65% of the amount of the loan. Fill in the payoff matrix below as in Exhibit 11-21. Which model threshold should David and the data scientist use?

Required

1. Calculate the model payoffs for both the worst- and best-case scenarios for each cutoff.
2. If you were David, how would this analysis impact your decision making?

Transcribed Image Text:

EXHIBIT 11-19 Actual Outcomes Confusion Matrix at Cutoff Value of 0.50 for Pruned Decision Tree Validation Sample Default Repay Total Predicted Outcomes Default (Do Not Invest in Loan) True Positives (A) 1 False Positives (Y) 0 1 Repay (Invest in Loan) False Negatives (Z) 1 True Negatives (B) 8 9 Total 2 8 10