Suppose that an IR system contains only 1000 documents. A query is known to generate 27 relevant documents as listed below:

{d1, d5, d7, d10, d88, d151, d200, d211, d250, d300, d399, d401, d405, d450, d473, d500, d501, d530, d545, d590, d600, d735, d700, d720, d800, d888, d900}.

Two different IR systems are used to retrieve ranked documents for this query. Each system only returns the top 10 ranked documents in order of ranking. Systems 1 and 2 each retrieves documents one at a time in the following order with all 10 documents eventually returned:

System 1: d122, d211, d150, d88, d37, d1, d501, d800, d201, d5.

System 2: d10, d700, d6, d250, d88,, d600, d59, d422, d500, d7.

Answer the following and show your work:

-Plot the Precision and the Recall graphs for each system as a function of the

number of documents returned (for 1 document returned, 2 documents

returned, etc).

-Plot the Precision versus Recall for systems 1 and 2 using these query results

as a function of the number of documents returned. Note that n1 is the value of

precision and recall for the first document, n2 for the 2 documents.

-=Which IR system is better? Justify your answer.

precision

n1 n2

n3 n4 recall

2. (2 points) What can be measured by a search engine? Precision or recall or both? Why?