Now that you have written functions for different steps of the model building process you will put it all together. You will write code that trains a model with hyperparameters you determine (you should do any tuning locally or in a notebook ie don't tune your model in gradescope since the autograder will likely timeout). It will take in the CLAMP training data, train a model then predict on a test set and output values from 0 to 1 for each row and our autograder will compare your predictions with the correct answers and to get credit you will need a roc auc score of .9 or higher on the test set (should not require much hyperparameter tuning for this dataset). This is basically a simulation of how your model would perform in the production system using batch inference.

Deliverables:

Make use of any of the techniques we covered in this project to train a model and return predicted probabilities for each row of the test set as a DataFrame with columns index (same as your index from the input test df) and malware_score (predicted probabilities).

Complete the train_model_return_scores function in task5.py

import numpy as np import pandas as pd

def train_model_return_scores(train_df_path,test_df_path) -> pd.DataFrame: # TODO: Load and preprocess the train and test dfs # Train a sklearn model using training data at train_df_path # Use any sklearn model and return the test index and model scores

# TODO: output dataframe should have 2 columns # index : this should be the row index of the test df # malware_score : this should be your model's output for the row in the test df test_scores = pd.DataFrame() return test_scores

Expert Answer