Part $2$ Compare the performances of different ML algorithms

a$)$ Randomly select two supervised learning modelling algorithms to test against

one another by running the following code. Make sure you enter your student ID

into the command set.seed$(.).$ Your $2$ ML approaches are given by myModels.

set.seed$($Enter your student ID$)$

models.list$1<-$ c$("$Logistic Ridge Regression",

"Logistic LASSO Regression",

"Logistic Elastic$-$Net Regression"$)$

models.list$2<-$ c$("$Classification Tree",

"Bagging Tree",

"Random Forest"$)$

myModels $<-$ c$($sample$($models$.$list$1,$ size $=1),$

sample$($models$.$list$2,$ size $=1))$

myModels $\%>\%$ data.frame

For each of your two ML modelling approaches, you will need to:

b$)$ Run the ML algorithm in R on the two training sets with Classification as the

outcome variable.

c$)$ Perform hyperparameter tuning to optimise the model:

$$ Outline your hyperparameter tuning$/$searching strategy for each of the ML

modelling approaches. Report on the search range$($s$)$ for hyperparameter

tuning, which $-$fold CV was used, and the number of repeated CVs $($if

applicable$),$ and the final optimal tuning parameter values and relevant CV

statistics $($i$.$e$.$ CV results, tables and plots$),$ where appropriate. If you are

using repeated CVs$,$ a minimum of $2$ repeats are required.

$$ If your selected tree model is Bagging, you must tune the nbagg, cp and

minsplit hyperparameters, with at least $3$ values for each.

$7|$ P a g e

$$ If your selected tree model is Random Forest, you must tune the num.trees

and mtry hyperparameters, with at least $3$ values for each.

$$ Be sure to set the randomisation seed using your student ID$.$

d$)$ Evaluate the predictive performance of your two ML models, derived from the

balanced and unbalanced training sets, on the testing set. Provide the confusion

matrices and report and interpret the following measures in the context of the

project:

$$ Overall Accuracy

$$ Precision

$$ Recall

$$ F$1-$score

Make sure you define each of the above metrics in the context of the study. Hint:

Use the help menu in R Studio on the confusionMatrix$(.)$ function to see how one

can obtain the precision, recall and F$1-$score metrics.

e$)$ Provide a brief statement on your final recommended model and why you have

chosen it$.$ This includes explaining which metric$($s$)$ you have used in making this

decision and why. Parsimony, and to a lesser extent, interpretability maybe

taken into account if the decision is close. You may outline your penalised model

estimates in the Appendix if it helps with your argument