Here is the question and following with data called Bank.csv

Using Python

Topic - KNN and NBC

KNN problems

Relatively young bank growing rapidly in terms of overall customer acquisition. The majority of these customers are liability customers (depositors) with varying sizes of relationship with the bank. The customer base of asset customers (borrowers) is quite small, and the bank is interested in expanding this base rapidly to bring in more loan business. In particular, it wants to explore ways of converting its liability customers to personal loan customers (while retaining them as depositors).

Campaign that the bank ran last year for liability customers showed healthy conversion rate of over 9% success. This has encouraged the retail marketing department to devise smarter campaigns with better target marketing. The goal is to use k-NN to predict whether new customer will accept loan offer. This will serve as the basis for the design of new campaign.

The file Bank.csvcontains data on 5000 customers. The data include customer demographic information (age, income, etc.), the customer's relationship with the bank (mortgage, securities account, etc.), and the customer response to the last personal loan campaign (Personal Loan). Among these 5000 customers, only 480 (=9.6%) accepted the personal loan that was offered to them in the earlier campaign.

Partition the data into training (60%) and validation (40%) sets.

a. Consider the following customer:

Age = 40, Experience = 10, Income = 84, Family = 2, CCAvg = 2, Education_1 = 0, Education_2 = 1, Education_3 = 0, Mortgage = 0, Securities Account = 0, CD Account = 0, Online = 1, and Credit Card = 1. Perform a k-NN classification with all predictors except ID and ZIP code using k= 1. Remember to transform categorical predictors with more than two categories into dummy variables first. Specify the success class as 1 (loan acceptance), and use the default cutoff value of 0.5. How would this customer be classified?

b. What is choice of kthat balances between overfitting and ignoring the predictor information?

c. Show the confusion matrix for the validation data that results from using the best k.

d. Consider the following customer: Age = 40, Experience = 10, Income = 84, Family= 2, CCAvg = 2, Education_1 = 0, Education_2 = 1, Education_3 = 0, Mortgage = 0, Securities Account = 0, CD Account = 0, Online = 1 and Credit Card = 1. Classify the customer using the best k.

e. Repartition the data, this time into training, validation, and test sets (50%:30%:20%). Apply the k-NN method with the kchosen above. Compare the confusion matrix of the test set with that of the training and validation sets. Comment on the differences and their reason.

Here is the data:

ID	Age	Experience	Income	ZIP Code	Family	CCAvg	Education	Mortgage	Personal Loan	Securities Account	CD Account	Online	CreditCard
1	25	1	49	91107	4	1.6	1	0	0	1	0	0	0
2	45	19	34	90089	3	1.5	1	0	0	1	0	0	0
3	39	15	11	94720	1	1	1	0	0	0	0	0	0
4	35	9	100	94112	1	2.7	2	0	0	0	0	0	0
5	35	8	45	91330	4	1	2	0	0	0	0	0	1
6	37	13	29	92121	4	0.4	2	155	0	0	0	1	0
7	53	27	72	91711	2	1.5	2	0	0	0	0	1	0
8	50	24	22	93943	1	0.3	3	0	0	0	0	0	1
9	35	10	81	90089	3	0.6	2	104	0	0	0	1	0
10	34	9	180	93023	1	8.9	3	0	1	0	0	0	0
11	65	39	105	94710	4	2.4	3	0	0	0	0	0	0
12	29	5	45	90277	3	0.1	2	0	0	0	0	1	0
13	48	23	114	93106	2	3.8	3	0	0	1	0	0	0
14	59	32	40	94920	4	2.5	2	0	0	0	0	1	0
15	67	41	112	91741	1	2	1	0	0	1	0	0	0
16	60	30	22	95054	1	1.5	3	0	0	0	0	1	1
17	38	14	130	95010	4	4.7	3	134	1	0	0	0	0
18	42	18	81	94305	4	2.4	1	0	0	0	0	0	0
19	46	21	193	91604	2	8.1	3	0	1	0	0	0	0
20	55	28	21	94720	1	0.5	2	0	0	1	0	0	1
21	56	31	25	94015	4	0.9	2	111	0	0	0	1	0
22	57	27	63	90095	3	2	3	0	0	0	0	1	0
23	29	5	62	90277	1	1.2	1	260	0	0	0	1	0
24	44	18	43	91320	2	0.7	1	163	0	1	0	0	0
25	36	11	152	95521	2	3.9	1	159	0	0	0	0	1
26	43	19	29	94305	3	0.5	1	97	0	0	0	1	0
27	40	16	83	95064	4	0.2	3	0	0	0	0	0	0
28	46	20	158	90064	1	2.4	1	0	0	0	0	1	1
29	56	30	48	94539	1	2.2	3	0	0	0	0	1	1
30	38	13	119	94104	1	3.3	2	0	1	0	1	1	1
31	59	35	35	93106	1	1.2	3	122	0	0	0	1	0
32	40	16	29	94117	1	2	2	0	0	0	0	1	0
33	53	28	41	94801	2	0.6	3	193	0	0	0	0	0
34	30	6	18	91330	3	0.9	3	0	0	0	0	0	0
35	31	5	50	94035	4	1.8	3	0	0	0	0	1	0