Answered step by step
Verified Expert Solution
Question
1 Approved Answer
1 Rank-based Evaluation Metrics, MAP@K, MRR@K [20pts] Assume you have three queries, and the ranking results that a system in response to these three queries
1 Rank-based Evaluation Metrics, MAP@K, MRR@K [20pts] Assume you have three queries, and the ranking results that a system in response to these three queries are as follows: Ranking 1 in response to query 1 is: d1, d2, d3, d4, d5, d6, d7, d8, d9, d10. Here only d1, d3, d6, d7, and d10 are relevant (relevance is binary, i.e., either 1 if relevant or 0 if non-relevant) in response to query 1 . Ranking 2 in response to query 2 is: d3,d8,d7, d1,d2,d4,d5,d9,d10,d6. Here only d 3 and d5 are relevant in response to query 2 . Ranking 3 in response to query 3 is: d7, d6, d5, d3, d2, d1, d9, d10, d4, d8. Here only d1, d9, and d8 are relevant in response to query 3 . Answer the questions below: (a) Compute the scores for these metrics: AP@5 (Average Precision @5),AP@10 for each query; RR@5 (Reciprocal Rank score @5),RR@10 for each query. (b) Compute the scores for these metrics: MAP@5 (Mean Average Precision @5),MAP@10, MRR@5 (Mean Reciprocal Rank score@5), MRR@10 for this system
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started