Which of the following is NOT used for the test or validation error?

a) Help fit a method with a fixed set of tuning parameters. b) Find the best values for tuning parameters in a particular method.

c) Compare different methods. d) Provide a measure of uncertainty for future predictions.

In which of the following situations does overfitting become likely?

a) A method is not flexible enough. b) A method is too flexible. c) Data have little variability around the true structure.

d) The sample size is very small. e) The sample size is very large.