In the context of the Confusion matrices, define these concepts and explain how they are used in the evaluation of a classifier: CA. Accuracy, CB. Precision, CC. Recall, CD. False positive rate.

Provide the following information:

Concept CA is defined as ... and is used to ... (2 marks) Concept CB is defined as ... and is used to ... (2 marks) Concept CC is defined as ... and is used to ... (2 marks) Concept CD is defined as ... and is used to ... (2 marks)

Consider the business case explained at the top of this paper.

Your team has decided to construct a classifier using a discretised label. They have tried three different models. Now they have asked you to select the best performing model based on the supplied evaluation results (in your answers refer to Figures 4A, 4B, ...).

PART A: The following four operators were used in the process. Very briefly explain the aims (what) and the specific reasons (why) of using the following operators in that process:

O1. The aim of "Set Role" is to ... because ... (2 marks) O2. The aim of "Discretize" is to ... because ... (2 marks) O3. The aim of "Nominal to Binomial" is to ... because ... (2 marks) O4. The aim of "Cross Validation" is to ... because ... (2 marks)

52.50!) 501000 4? 500 4500:} 2.50!) 40.000 31500 35.00!) 32.500 30.000 21500 25.00:) 22.500 20.000 1150!) 15.000 12.500 10.000 1".500 uric: 51000 2.500 0 moderate expensive Figure 4 (Part A): Class distribution of the label attribute, discretised at $50K k-NN k=7 Performance accuracy: 94.44% +/- 0.13% (micro average: 94.44%) kappa: 0.470 +/- 0.007 (micro average: 0.470) AUC: 0.958 +/- 0.003 (micro average: 0.958) (positive class: expensive) Decision Tree Performance accuracy: 93.30% +/- 0.24% (micro average: 93.30%) kappa: 0.694 +/- 0.026 (micro average: 0.694) AUC: 0.968 +/ 0.008 (micro average: 0.968) (positive class: expensive) Logistic Regression accuracy: 93.90% +/- 0.22% (micro average: 93.90%) kappa: 0.437 +/- 0.029 (micro average: 0.437) AUC: 0.929 +/- 0.005 (micro average: 0.929) (positive class: expensive) Figure 4 (Part B): Performance of three classification models