Subject: Special Topics $-$ Machine Learning

Assignment #$1$: Understanding Performance Metrics: Calculating Accuracy, Precision, Recall, and F$1-$score.

Objective:

The aim of this assignment is to provide students with hands$-$on experience in calculating key performance metrics: Accuracy, Precision, Recall, and F$1-$score$-$from a confusion matrix. Additionally, students will utilize Excel functions to automate the calculation process, enhancing their proficiency in both manual calculation and utilizing digital tools for analysis.

Assignment Tips:

Ensure a solid understanding of confusion matrices and the concepts of true positives, true negatives, false positives, and false negatives.

Familiarize yourself with the formulas for Accuracy, Precision, Recall, and F$1-$score and how they are derived from the confusion matrix.

Practice using Excel functions such as SUM, and AVG, to efficiently calculate the metrics.

Double$-$check your calculations and formula implementations to avoid errors.

Pay attention to the interpretation of each metric and understand their significance in evaluating classification model performance.

Page $1$ of $3$

Task $1$: Understanding Confusion Matrices

Define a confusion matrix and explain its components.

Provide a hypothetical scenario where a confusion matrix would be utilized $($e$.$g$.,$ diagnosing medical conditions$).$

Task $2$: Manual Calculation of Metrics

Given the following confusion matrix values, manually calculate the Accuracy, Precision, Recall, and F$1-$score:

$>$ True Positives $(TP)=120$

$>$ True Negatives $(TN)=80$

$>$ False Positives $(FP)=20$

$>$ False Negatives $(FN)=10$

Task $3$: Excel Implementation

Create an Excel spreadsheet to automate the calculation of Accuracy, Precision, Recall, and F$1-$score using formulas.

Input the values from the confusion matrix and ensure that the spreadsheet accurately calculates each metric.

Task $4$: Interpretation

Interpret the calculated metrics in the context of the scenario provided.

Discuss the strengths and limitations of each metric and when they might be most useful.

Task $5$: Real$-$world Application

Research and present a real$-$world example where these metrics are employed to evaluate the performance of a classification model.

Describe the scenario and explain how the metrics were utilized to assess the model's effectiveness.

Submission Guidelines:

Submit both a Word document and an Excel spreadsheet.

The Word document should contain detailed explanations, calculations, interpretations, and the real$-$world application.

The Excel spreadsheet should demonstrate the automated calculation of metrics using appropriate formulas.

Ensure clarity and organization in your submission.

Deadline: $11$ May $2024$

Grading Rubric:

Accuracy of manual calculations

Correct implementation of Excel functions

Depth of interpretation and analysis

Integration of real$-$world example

Adherence to submission guidelines