Our goal for this assignment is to write a function in R that carries out a parameter search for a support vector machine using the approach described above. The function will take as arguments a training dataset, the name of the model to build, and a list of parameters to search. Its skeleton will look like this:

param.search <- function(formula, data, params) { # split the data into training and validation sets # carry out the parameter search, evaluating the error rate (misclassification rate) on the validation set # return the best set of parameters, errors, and model }

It will return a list containing the best values of the parameters, the error, and the best model. Below is some sample code showing how we would use it.

params <- list(cost=c(0.001, 0.01, 0.1, 1, 10, 100), kernel=c("linear","radial")) res <- param.search(class~., heart.split$train, params) print(res$params) print(res$error) print(summary(res$model))

If that's enough for you to write it, have at it! The rest of the commentary below consists of hints and R info. The first challenge to deal with is to generate the grid of parameter values for each model from the list that is passed in. This is simple, though, and fortunately, we've already seen how to do it. Last week's code included

tuning.grid <- expand.grid(interaction.depth = 1:6, n.trees = (1:10)*500, shrinkage = c(0.001, 0.01, 0.1), n.minobsinnode = 10)

and expand.grid() is part of the base R distribution. Experiment with this function and convince yourself it generates a data.frame with one row for each combination of specified parameter values. The column names in the data.frame are the names of the arguments needed to pass to the model, and the values are in each row. We can build a model for each combination of parameters simply by looping over the rows of this data.frame or by applying a function to each row if you know how to do that. The next challenge may be how to call a function to build a model when all the info you need for the function, including the argument names, is stored in variables. Once again, R to the rescue! do.call(f, args) takes two arguments, a function f and a list of arguments. Looking at this week's svm code, we build a model like

glass.svm.poly <- svm(Type~., data=glass.split$train, scale=FALSE, kernel="polynomial", degree=4)

but we could just have just as well have done

glass.svm.poly <- do.call(svm, list(formula=Type~., data=glass.split$train, scale=FALSE, kernel="polynomial", degree=4))

and get the same result. We can see that R turns the list entries into arguments and even uses the names from the list for the parameter names. So we can use expand. grid to generate the search grid, loop over it, and then in the loop use do. call to build a model with the correct parameters being passed in. After that, predict the validation set and be sure to save the model with the lowest error rate.