3.11 Exis 191 Table 3.7. Comparing the test c ry of decision trees Trand T1 Accuracy Data Set TOT 0.86 0.97 0.77 0.84 12. Consider a labeled data set containing 100 data instances, which is randomly partitioned into two sets A and B ench containing 50 instances. We use Ans the training set to lenen two decision trees, 7o with 10 leaf nodes and Too with 100 leaf nodes. The accuracies of the two decision trees on data sets A and B are shown in Table 3.7. (a) Based on the accuracies shown in Table 3.7, which classification motel would you expect to have better performance on unseen bestances? (b) Now, you tested T. and T20 on the entire dala set (A + B) and found that the classification accuracy of Tuo on data we ( A B ) in O.RS, where the classification accuracy of T100 on the data set (A + B) is 0.87. Basel on this new information and your observations from Table 3.7, which classification model would you finally choose for classification? 13. Consider the following approach for testing whether a classifier A beats another classifier B. Let N be the size of a given dataset, a be the accuracy of classifice A. Pe be the accuracy of classifier 8, and p = (PAPs/2 be the average accuracy for both classiliers. To test whether classifier A is significantly better than B, the following Z-statistic is used: z = P4-PB Classifier A is assumed to be better than classifier BilZ 1.96. Table 3.8 compares the accuracies of three different classifiers, decision tree classifiers, naive Bayes classifiers, and support vector machines, on various data sets. (The latter two classifiers are described in Chapter 4.) Summarize the performance of the classifiers given in Table 3.8 using the following 3 x 3 table: Decision tree Naive Bayes win-loss-draw Support vector machine 0 - 0 - 23 T0-0-23T Decision tree Naive Bayes Support vector machine 3.11 Exis 191 Table 3.7. Comparing the test c ry of decision trees Trand T1 Accuracy Data Set TOT 0.86 0.97 0.77 0.84 12. Consider a labeled data set containing 100 data instances, which is randomly partitioned into two sets A and B ench containing 50 instances. We use Ans the training set to lenen two decision trees, 7o with 10 leaf nodes and Too with 100 leaf nodes. The accuracies of the two decision trees on data sets A and B are shown in Table 3.7. (a) Based on the accuracies shown in Table 3.7, which classification motel would you expect to have better performance on unseen bestances? (b) Now, you tested T. and T20 on the entire dala set (A + B) and found that the classification accuracy of Tuo on data we ( A B ) in O.RS, where the classification accuracy of T100 on the data set (A + B) is 0.87. Basel on this new information and your observations from Table 3.7, which classification model would you finally choose for classification? 13. Consider the following approach for testing whether a classifier A beats another classifier B. Let N be the size of a given dataset, a be the accuracy of classifice A. Pe be the accuracy of classifier 8, and p = (PAPs/2 be the average accuracy for both classiliers. To test whether classifier A is significantly better than B, the following Z-statistic is used: z = P4-PB Classifier A is assumed to be better than classifier BilZ 1.96. Table 3.8 compares the accuracies of three different classifiers, decision tree classifiers, naive Bayes classifiers, and support vector machines, on various data sets. (The latter two classifiers are described in Chapter 4.) Summarize the performance of the classifiers given in Table 3.8 using the following 3 x 3 table: Decision tree Naive Bayes win-loss-draw Support vector machine 0 - 0 - 23 T0-0-23T Decision tree Naive Bayes Support vector machine
