Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Task 7 Evaluating the intrusion detection system Our final task is to evaluate how well our intrusion detection system works. This is done by

imageimageimage

Task 7 Evaluating the intrusion detection system Our final task is to evaluate how well our intrusion detection system works. This is done by calculating some performance metrics. For our evaluation, we will calculate four metrics - accuracy, precision, recall and F1 score. The calculations of these metrics are all available from this page (you can do your own research in addition to reading this): https://en.wikipedia.org/wiki/F-score The accuracy measures how well the intrusion detection system has performed. The precision measures the ratio of intrusions detected out of all detections. The recall measures the ratio of intrusions detected out of all intrusions. F1 score measure is a balanced representation of both precision and recall. For example, high precision means you were able to detect a high number of intrusions. On the other hand, high recall means you have a low number of wrongly classified benign data. In case of high precision and low recall, it means you have likely just labelled most data as intrusion (thus, many false negatives). In case of low precision and high recall, it means many intrusions bypassed the intrusion detection system. So, a good balance between the two are essential in practice, which is measured by F1 Score metric. You already have two lists - detected and benign, from Task 6. So you can use these two lists to calculate these metrics. Write a function result_analysis (data, detected, benign) that prints the statistical analysis results for your intrusion detection system. The outputs should be, in the order: Accuracy : label "Accuracy" with the text width being 10, followed by colon ":", followed by the result formatted to 4 decimal places with the text width being 10 also. Precision, Recall and F1 Score are printed in this order with the same formatting. Using the default values (i.e., upper_sd and lower_sd set to 3, ceiling set to 0.01) with the dataset proj1_data0.csv, you should get the following output. Accuracy : Precision : Recall 0.5464 0.0739 0.9997 F1 Score : 0.1376 Figure 5. The evaluation of our IDS using default values In fact, the result is pretty bad! - The F1 Score of 0.1 is actually worse than a random guess (e.g., a coin-flip method will achieve 0.5 F1 Score)! Well thankfully, we can improve our IDS! For enthusiastics only --- Following the specifications from the project 1 instruction, write a function result_analysis (data, detected, benign) that prints the statistical analysis results for your intrusion detection system. More details are found in the project 1 instruction sheet. Submit ALL functions necessary to run the result_analysis function, including imports. For example: Test data = get_data("proj1_data0.csv") stat = ecu_summary(data, False) detected, benign result_analysis(data, detected, benign) = Result Accuracy : Precision : intrusion_detection(data, stat) Recall : F1 Score : 0.5464 0.0739 0.9997 0.1376

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image_2

Step: 3

blur-text-image_3

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Managerial Accounting

Authors: Ray H. Garrison, Eric W. Noreen, Peter C. Brewer

12th Edition

978-0073526706, 9780073526706

More Books

Students also viewed these Programming questions

Question

rationality in artifical intelligence

Answered: 1 week ago