While the .632 bootstrap approach is useful for obtaining a reliable estimate of model accuracy, it has a known limitation. Consider a two-class problem, where there are equal number of positive and negative examples in the data.
Suppose the class labels for the examples are generated randomly. The classifier used is an un-pruned decision tree (i.e., a perfect memorizer). Determine the accuracy of the classifier using each of the following methods.
(a) The holdout method, where two-thirds of the data are used for training and the remaining one-third are used for testing.
(b) Ten-fold cross-validation.
(c) The .632 bootstrap method.
(d) From the results in parts (a), (b), and (c), which method provides a more reliable evaluation of the classifier's accuracy?