Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for...
Fantastic news! We've Found the answer you've been seeking!
Question:
Transcribed Image Text:
Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for relevance to a query. A 3-point relevance scale was used, so relevant documents have been divided into Perfect, Good and just Relevant results. Weights for these levels are shown below for NDCG(Normalized Discounted Cumulative Gain): Perfect 3 Good 2 Relevant 1 Non-relevant 0 Consider the result lists retrieved for the three different information needs shown as below respectively: Result Q Result_Q2 Result Q3 = <3,0,2,2,0> = = <3, 2, 2, 2, 0, 2, 0, 1 > < 0,2,0,3> 1. Assume there are totally 10 relevant documents in the collection. What are the precision and recall for result list Result-Q2? Draw the interpolated precision-recall curve. 2. What is the precision @4 of each result list? (4) (3) 3. What is the average precision of each result list? and what is MAP for this IR system if there are only these three information needs in the test collection? (4) 4. What is the perfect ranking for Result Q = < 3,2,2,2,0, 2, 0, 1 >? And calculate the Ideal Discounted Cumulative Gain (DCG) for this set of documents. (5) 5. To measure/evaluate information retrieval (IR) effectiveness, what are the three elements required for a test collection, so the performance of the IR system could be compared? (3) 6. What is Kappa Measure? (2) 7. For a particular information need if Judge 1 rated the relevance of a set of 5 documents as Result = < < R, N, R, R, N > and Judge 2 rated as Result2 = < R, R, R, N, N >. Calculate the Kappa measure if the expected chance agreement ratio P(E) is 0.5. (4) Suppose we have a collection of 20 documents, d, d2,..., d20, which have been judged for relevance to a query. A 3-point relevance scale was used, so relevant documents have been divided into Perfect, Good and just Relevant results. Weights for these levels are shown below for NDCG(Normalized Discounted Cumulative Gain): Perfect 3 Good 2 Relevant 1 Non-relevant 0 Consider the result lists retrieved for the three different information needs shown as below respectively: Result Q Result_Q2 Result Q3 = <3,0,2,2,0> = = <3, 2, 2, 2, 0, 2, 0, 1 > < 0,2,0,3> 1. Assume there are totally 10 relevant documents in the collection. What are the precision and recall for result list Result-Q2? Draw the interpolated precision-recall curve. 2. What is the precision @4 of each result list? (4) (3) 3. What is the average precision of each result list? and what is MAP for this IR system if there are only these three information needs in the test collection? (4) 4. What is the perfect ranking for Result Q = < 3,2,2,2,0, 2, 0, 1 >? And calculate the Ideal Discounted Cumulative Gain (DCG) for this set of documents. (5) 5. To measure/evaluate information retrieval (IR) effectiveness, what are the three elements required for a test collection, so the performance of the IR system could be compared? (3) 6. What is Kappa Measure? (2) 7. For a particular information need if Judge 1 rated the relevance of a set of 5 documents as Result = < < R, N, R, R, N > and Judge 2 rated as Result2 = < R, R, R, N, N >. Calculate the Kappa measure if the expected chance agreement ratio P(E) is 0.5. (4)
Expert Answer:
Answer rating: 100% (QA)
Here are the answers to the questions 1 Result Q has 2 Perfect results and 2 Good results out of a t... View the full answer
Related Book For
Understanding Basic Statistics
ISBN: 9781111827021
6th Edition
Authors: Charles Henry Brase, Corrinne Pellillo Brase
Posted Date:
Students also viewed these finance questions
-
Research Assignment Using KeyCite on Westlaw A. Locate MacPhee v. Nicholson , 459 F.3d 1323. What status flag is displayed? Select the flag. Locate the first reference to which you are directed. Give...
-
Python and most Python libraries are free to download or use, though many users use Python through a paid service. Paid services help IT organizations manage the risks associated with the use of...
-
answer all questions as instructed below. attend all questions. 4 Computer Vision (a) Explain why such a tiny number of 2D Gabor wavelets as shown in this sequence are so efficient at representing...
-
In Exercises 1126, determine whether each equation defines y as a function of x. x + y = 25
-
A drug user responded to an ad placed by a DEA informant in a drug-culture magazine. He later flew from Colorado to Maryland, where he bought some 1-phenyl-2-propanone (P2P) from the informant. The...
-
Wide beam equation. Consider a plate of length a and width b that bends in only one direction such that w(x,y) = f(x). (a) Derive expressions for the bending moments, shear forces, and applied load...
-
The Thermo-Bond Manufacturing Company maintains its fixed-asset records on its computer. The fixed-asset master file includes the following data items: Required Refer to Table 9-7, which describes...
-
Stark Company has five employees. Employees paid by the hour receive a $10 per hour pay rate for the regular 40-hour workweek plus one and one-half times the hourly rate for each overtime hour beyond...
-
a vessel, 7 . 5 meters high contains 2 5 0 kg of a gas at 5 8 0 kpa gauge and 2 7 degrees celsius. atmospheric pressure is 1 0 1 . 3 2 5 kPa. ( a ) what is the diameter of the vessel if 1 kg of the...
-
Reverend Peter Wilson qualifies as a minister for income tax purposes. He receives a $75,000 annual salary and a $32,000 housing allowance. He pays $24,000 (including utilities) per year for the...
-
Agostino purchased five crypto coins for $1,300 in October of 2021. He sold two of the coins for $650 in May of 2022. He later sold the remaining three coins for $695 in September of 2022. What is...
-
For each of the following, find x in terms of y: 5 4x + 3y = 2x + 21y
-
Which feature in Simucase assists faculty with providing feedback to students? A. Faculty Dashboard B. Part Task Trainer C. Video Library D. Assessment Form
-
Which suppliers provide more products than the average for all suppliers (who provide products; do not include suppliers who do not provide any products)? Your query should show the supplier company...
-
Time Remaining 1 hour 24 minutes 38 seconds01:24:38\ Item 8\ Time Remaining 1 hour 24 minutes 38 seconds01:24:38\ Ethan (single) purchased his home on July 1, 2013. He lived in the home as his...
-
Which VMware feature maintains a live copy of a VM on another host to fail over to if the host the primary copy is running on goes down? HA O VM affinity 0000 O DRS O FT
-
Expenditure programs for the poor, commonly known as welfare programs or social safety nets, are government efforts intended to offer monetary aid, assistance, and resources to individuals and...
-
Which of the following statements is false? a. Capital leases are not commonly reported in a Capital Projects Fund. b. A governmental entity may report a Capital Project Fund in one year but not the...
-
Use chloroform to extract acetone from water. Equilibrium data are given in Table 13-6. Find number of equilibrium stages required for a countercurrent cascade if feed is \(1000.0 \mathrm{~kg} /...
-
We are extracting pyridine from \(500 \mathrm{~kg} / \mathrm{h}\) of a feed that is \(15.0 \mathrm{wt} \%\) pyridine and \(85.0 \mathrm{wt} \%\) water using \(225 \mathrm{~kg} / \mathrm{h}\) of pure...
-
Equilibrium for extraction of acetic acid from 3-heptanol into water at \(25^{\circ} \mathrm{C}\) is \(\mathrm{y}=1.208 \mathrm{x}\), where \(\mathrm{y}=\) weight fraction acetic acid in water and...
Study smarter with the SolutionInn App