Question 1

Supervised learning refers to learning methods that:

Group of answer choices

a. use the outcome in testing the method to assess predictive performance (calculate test error)

b. use only the covariates or X values to train the model

c. use both the outcome and the X values to test the model to assess predictive accuracy

d. use both the outcome and the X values to train the model before testing it

Question 2

Assume that you work for a network security company and are responsible for creating a prediction model which uses level of current activity to predict occurrence of cyber attacks. It is widely known that the relationship between current activity and occurrence of cyber attacks is nonlinear. Which model do you think will be more favorable? Assume thatY

Yis number of cyber attacks,X

Xis current activity level, and

is measurement error

Y=

0

+

1

X+

2

X

2

+

Y=0+1X+2X2+Model (1)

Y=0+1X+Model (2)

Group of answer choices

a. I would expect Model 1 and Model 2 to perform equally well - there is bias and variance to consider, one will be less biased with more variance, and one will be more biased with less variance

b. I would expect the linear model with no quadratic term (Model 2) to perform better. Quadratic terms create more variability and thus are not preferred.

c. I would expect Model 1 to perform better because it is widely known that the relationship is non-linear. The relationship in Model 2 is strictly linear, so it may underperform in this setting.

d. We do not have enough information to answer this problem. Nothing in the problem statement helps me decide.

Question 3

What are the two main goals of estimating the function f in the following equation, and when are they used?

Y=f(X)+

Y=f(X)+

Group of answer choices

a. The main goals are inference and description. The inference is useful for determining when effects exist, and description describes the effects.

b. The main goals are inference and prediction. Inference allows testing whether effects are important and describes their form or shape, while prediction is used to accurately predict the outcome of interest (Y)

c. The main goals are description and prediction. The description describes the effects of the covariates and prediction aims to accurately predict the outcome

d. The main goal is prediction. This class is about accurately predicting an outcome Y from covariate/input/feature values X.

Question 4

Reducible error is so termed because:

Group of answer choices

a. The error can be reduced by taking more accurate measurements of the outcome $Y$

b. The error can be reduced by improving the model, either through use of non-linear forms or including more relevant predictors

c. The error cannot be reduced - it is another name for measurement error, or the epsilon part of the "correct" model

Question 5

Irreducible error is so termed because:

Group of answer choices

a. It is error that may not be reduced - it will persist and stay constant, regardless of model quality

b. it is the error which can be decreased by reducing either bias or variance, or both!

c. It is the error that can be decreased by adding very non-linear terms, or creating an incredibly flexible model

Question 6

What are the benefits of using a parametric estimation approach? Pick the most complete answer.

Group of answer choices

a. The model is very flexible, and is good at prediction due to it's fixed form.

b. The model is often a simplification of reality, and is easy to interpret.

c. The model is often a simplification of reality. It is usually easier to interpret than a nonparametric method, because it uses a very specific form for the f(X) function. The estimation problem is reduced to estimating a fixed number of parameters. These models also tend to make inference easier because of their fixed form.

d. The model is good at inference due to having a very specific, fixed form.

Question 7

What are the benefits of using a non-parametric estimation approach? Pick the most complete answer.

Group of answer choices

a. A non-parametric approach is beneficial because it is very flexible and can provide extremely accurate predictions, while the constraints may be used to assure the model is not over-fit to the training data.

b. A non-parametric approach is beneficial because it can result in a very flexible and accurate prediction.

c. A non-parametric approach is beneficial because it is easy to conduct statistical inference using them.

d. A non-parametric approach is beneficial because the results are very easy to interpret as the estimation is reduced to a fixed number of parameters.

Question 8

Pick the best definition of overfitting.

Group of answer choices

a. Overfitting is what has occurred when a model has more than 5 predictors. We can expect the model to have high variability.

b. Overfitting is what has occurred when the model includes too many covariates or effects so that it fits the training data too closely. This leads the model to fit "the noise" and not just the signal, which reduces prediction accuracy on new data.

c. Overfitting is what has occurred when a model estimates parameters over a threshold of significance. The resulting model will be highly biased for predicting new data.

d. Overfitting is what has occurred when our model predicts the test data (new data) very accurately. This model is likely too variable, even though it may have low bias.

Question 9

Comment on the (general) shape of test error (the ability of a model to predict new data) as model flexibility increases from left to right. What happens as we consider more and more flexible models?

Group of answer choices

a. Test error will continue to decrease as the model flexibility increases.

b. Test error will first decrease as we increase model flexibility, but eventually, as the variance increases more relative to decreases in bias, the test error will increase

c. Test error will increase at first, due to variability. Eventually, bias will become negligible and the test error will thus become very small.

d. Test error will not change regardless of the flexibility level of the model.

Question 10

In the classification setting, what quantity is equivalent to the irreducible error in regression:Var()

Var()

Group of answer choices

a. The Bayes' error rate.

b. The classification rate of the model.

c. The difference between 1 and the max of the probabilities of group membership.

d. There is no analog to the irreducible error rate in classification problems.