50 pts. Q1The manager wants to see the ROC curve for each detector to evaluate their sensitivity based on different response criteria. He selected response criteria as follows: 45% of defectiveness as a neutral criterion, 25% of defectiveness as a liberal criterion and 65% of defectiveness as a conservative criterion. Use table 1 to: 1. (20pts) Calculate the hit alarm probabilities for each criterion and for each detector (20pts) Calculate the false alarm probabilities for each criterion and for each detector III. (10pts) Draw ROC curve of each detector to help the manager. Discuss your results. II. 20 pts. Q2 Once the manager determines the most sensitive agent, he also wants to know the optimal policy for response criterion. He knows each non defective item will bring around 60 KD net profit to facility, he also knows that if he can detect an defective item before it leaves from the facility, he can take an immediate action to withdraw it back with a cost of 30 KD per item and, thus the facility will gain a loyal customer; if he cannot detect an defective item before it leaves from the facility, it will result the loss of one loyal customer plus loss of prestige costing extra 30 KD per customer, if he can avoid calling a non-defective item as defective, he can save cost of taking action for defective items plus the facility will gain extra 20 KD per customer. 30 pts. Q3 The following results in table 2 were obtained from an experiment of a training session for the detection task by letting agents respond with confidence level. Calculate the necessary probabilities for conservative and for liberal response criteria and draw the ROC curve for each subject. Consider a neutral criterion of 40% unhappiness as a reference point to discriminate between signal and noise. Table 1. Percent defectiveness data collected from fitting test comparing to agent detection responses Agent 1 Agent 2 Agent 3 Percent Percent Percent Percent Percent Percent defectiveness defectiveness defectiveness defectiveness defectiveness defectiveness for YES for NO for YES for NO for YES for NO response response response response response response 60 29 43 57 44 100 65 7 29 46 21 51 67 24 53 55 92 54 75 47 36 51 71 55 55 0 92 38 28 54 85 50 64 27 59 44 69 53 100 34 46 58 68 48 58 47 67 63 90 11 83 37 17 53 19 100 41 19 56 70 76 15 60 45 49 67 38 9 79 42 67 37 38 46 42 76 38 43 80 27 42 36 88 26 83 21 39 52 89 58 65 63 14 42 76 17 56 24 45 42 54 39 69 36 36 22 5 70 76 67 38 82 62 86 32 52 19 93 36 45 39 85 69 66 48 37 67 68 38 83 30 100 65 91 52 34 30 97 4 44 29 100 80 73 95 33 36 68 41 48 18 58 S6 96 39 50 66 31 18 54 57 57 28 55 30 32 20 46 9 27 53 83 45 56 4 35 67 64 73 32 45 18 57 17 56 34 100 65 88 51 32 69 54 100 66 6 39 46 61 43 53 26 19 54 42 48 98 97 Table 2. Results of the experiment of a training session on three subjects Event Actual Subjects' Response Percent No. Defectiveness S1 S2 S3 1 Nondefective Nondefective Uncertain 10 2 Nondefective Uncertain Uncertain 40 3 Uncertain Uncertain Uncertain 55 4 defective Uncertain Uncertain 65 5 defective defective Uncertain 70 6 defective defective defective 80 7 Nondefective defective defective 95 8 Uncertain Nondefective Uncertain 55 9 Uncertain Uncertain Uncertain 60 10 defective defective defective 85 11 Nondefective Nondefective Nondefective 5 12 Nondefective Nondefective Nondefective 10 13 Uncertain Uncertain Uncertain 65 14 Uncertain defective defective 45 15 Nondefective Nondefective defective 35 16 Uncertain defective Uncertain 45 17 Uncertain defective Uncertain 50 18 defective defective Uncertain 40 19 Nondefective Uncertain Uncertain 55 50 pts. Q1The manager wants to see the ROC curve for each detector to evaluate their sensitivity based on different response criteria. He selected response criteria as follows: 45% of defectiveness as a neutral criterion, 25% of defectiveness as a liberal criterion and 65% of defectiveness as a conservative criterion. Use table 1 to: 1. (20pts) Calculate the hit alarm probabilities for each criterion and for each detector (20pts) Calculate the false alarm probabilities for each criterion and for each detector III. (10pts) Draw ROC curve of each detector to help the manager. Discuss your results. II. 20 pts. Q2 Once the manager determines the most sensitive agent, he also wants to know the optimal policy for response criterion. He knows each non defective item will bring around 60 KD net profit to facility, he also knows that if he can detect an defective item before it leaves from the facility, he can take an immediate action to withdraw it back with a cost of 30 KD per item and, thus the facility will gain a loyal customer; if he cannot detect an defective item before it leaves from the facility, it will result the loss of one loyal customer plus loss of prestige costing extra 30 KD per customer, if he can avoid calling a non-defective item as defective, he can save cost of taking action for defective items plus the facility will gain extra 20 KD per customer. 30 pts. Q3 The following results in table 2 were obtained from an experiment of a training session for the detection task by letting agents respond with confidence level. Calculate the necessary probabilities for conservative and for liberal response criteria and draw the ROC curve for each subject. Consider a neutral criterion of 40% unhappiness as a reference point to discriminate between signal and noise. Table 1. Percent defectiveness data collected from fitting test comparing to agent detection responses Agent 1 Agent 2 Agent 3 Percent Percent Percent Percent Percent Percent defectiveness defectiveness defectiveness defectiveness defectiveness defectiveness for YES for NO for YES for NO for YES for NO response response response response response response 60 29 43 57 44 100 65 7 29 46 21 51 67 24 53 55 92 54 75 47 36 51 71 55 55 0 92 38 28 54 85 50 64 27 59 44 69 53 100 34 46 58 68 48 58 47 67 63 90 11 83 37 17 53 19 100 41 19 56 70 76 15 60 45 49 67 38 9 79 42 67 37 38 46 42 76 38 43 80 27 42 36 88 26 83 21 39 52 89 58 65 63 14 42 76 17 56 24 45 42 54 39 69 36 36 22 5 70 76 67 38 82 62 86 32 52 19 93 36 45 39 85 69 66 48 37 67 68 38 83 30 100 65 91 52 34 30 97 4 44 29 100 80 73 95 33 36 68 41 48 18 58 S6 96 39 50 66 31 18 54 57 57 28 55 30 32 20 46 9 27 53 83 45 56 4 35 67 64 73 32 45 18 57 17 56 34 100 65 88 51 32 69 54 100 66 6 39 46 61 43 53 26 19 54 42 48 98 97 Table 2. Results of the experiment of a training session on three subjects Event Actual Subjects' Response Percent No. Defectiveness S1 S2 S3 1 Nondefective Nondefective Uncertain 10 2 Nondefective Uncertain Uncertain 40 3 Uncertain Uncertain Uncertain 55 4 defective Uncertain Uncertain 65 5 defective defective Uncertain 70 6 defective defective defective 80 7 Nondefective defective defective 95 8 Uncertain Nondefective Uncertain 55 9 Uncertain Uncertain Uncertain 60 10 defective defective defective 85 11 Nondefective Nondefective Nondefective 5 12 Nondefective Nondefective Nondefective 10 13 Uncertain Uncertain Uncertain 65 14 Uncertain defective defective 45 15 Nondefective Nondefective defective 35 16 Uncertain defective Uncertain 45 17 Uncertain defective Uncertain 50 18 defective defective Uncertain 40 19 Nondefective Uncertain Uncertain 55