A bank uses a composite score based on the customer's age, average checking account balance, having a mortgage, job status, marital status, and level of education to predict the probability (between 0 and 1) that the customer will default a loan. The bank must choose a cutoff for loan decisions. Customers with predicted probabilities of default below the cutoff will receive loans and those above the cutoff will not. If the bank grants a loan that is later defaulted, we consider it a Class 1 Error and the bank loses the remaining loan balance. On the other hand, if the bank does not grant a loan to a customer who would have paid the loan back, we consider it a Class 0 Error and the bank loses the interest they would have earned on the loan. Thus, both types of errors cost the bank money. The top graph below illustrates the tradeoff in the two kinds of classification errors. Unfortunately, no cutoff avoids both kinds of errors. The performance of a given cutoff score is sometimes characterized in terms of Sensitivity ( 1 - Class 1 error rate) and Specificity ( 1 - Class 0 error rate). The "receiver operating characteristic curve" or ROC curve, originally developed to measure the performance of radar systems, provides a useful way to depict the tradeoff between the two kinds of errors. For the ROC, the Sensitivity is plotted against 1 - Specificity, which is the Class 0 Error Rate. Most desirable is high Sensitivity with a low Class 0 error rate (i.e., high Specificity). The diagonal line shows what the expected performance would be for guessing. Drag left and right in upper graph and observe the corresponding point in the ROC in the lower graph. () Cengage Learning. All Rights Reserved. 1. Which of the following is least accurate? a. For changes in the cutoff value, if the Class 1 Error rate stays the same, then the sensitivity stays the same. b. For changes in the cutoff value, as the Class 0 Error rate increases, the Sensitivity increases. c. As the cutoff value increases, the Class 0 Error rate decreases or stays the same. d. As the cutoff value increases, the Class 1 Error rate increases or stays the same. 2. For which approximate Cutoff Value below is the Sensitivity less than what would be expected for random guessing? a. 0.35 b. 0.46 c. 0.68 d. 1.00 3. Which of the following is true? a. Whenever the Class 0 Error rate changes, the Class 1 Error rate changes. b. Whenever the Cutoff Value changes, the Sensitivity changes. c. Whenever the Class 1 Error rate decreases, the Sensitivity increases. d. Whenever Sensitivity is above the diagonal line, guessing outperforms the classification model. A bank uses a composite score based on the customer's age, average checking account balance, having a mortgage, job status, marital status, and level of education to predict the probability (between 0 and 1) that the customer will default a loan. The bank must choose a cutoff for loan decisions. Customers with predicted probabilities of default below the cutoff will receive loans and those above the cutoff will not. If the bank grants a loan that is later defaulted, we consider it a Class 1 Error and the bank loses the remaining loan balance. On the other hand, if the bank does not grant a loan to a customer who would have paid the loan back, we consider it a Class 0 Error and the bank loses the interest they would have earned on the loan. Thus, both types of errors cost the bank money. The top graph below illustrates the tradeoff in the two kinds of classification errors. Unfortunately, no cutoff avoids both kinds of errors. The performance of a given cutoff score is sometimes characterized in terms of Sensitivity ( 1 - Class 1 error rate) and Specificity ( 1 - Class 0 error rate). The "receiver operating characteristic curve" or ROC curve, originally developed to measure the performance of radar systems, provides a useful way to depict the tradeoff between the two kinds of errors. For the ROC, the Sensitivity is plotted against 1 - Specificity, which is the Class 0 Error Rate. Most desirable is high Sensitivity with a low Class 0 error rate (i.e., high Specificity). The diagonal line shows what the expected performance would be for guessing. Drag left and right in upper graph and observe the corresponding point in the ROC in the lower graph. () Cengage Learning. All Rights Reserved. 1. Which of the following is least accurate? a. For changes in the cutoff value, if the Class 1 Error rate stays the same, then the sensitivity stays the same. b. For changes in the cutoff value, as the Class 0 Error rate increases, the Sensitivity increases. c. As the cutoff value increases, the Class 0 Error rate decreases or stays the same. d. As the cutoff value increases, the Class 1 Error rate increases or stays the same. 2. For which approximate Cutoff Value below is the Sensitivity less than what would be expected for random guessing? a. 0.35 b. 0.46 c. 0.68 d. 1.00 3. Which of the following is true? a. Whenever the Class 0 Error rate changes, the Class 1 Error rate changes. b. Whenever the Cutoff Value changes, the Sensitivity changes. c. Whenever the Class 1 Error rate decreases, the Sensitivity increases. d. Whenever Sensitivity is above the diagonal line, guessing outperforms the classification model
