import nltk

nltk.download('punkt')

from nltk import word_tokenize

sentences=["Sheikh Mohamed bin Zayed Al Nahyan GCMG, colloquially known by his initials as MBZ, is the third president of the United Arab Emirates and the ruler of Abu Dhabi. ",

"On April 13, 2007, Google reached an agreement to acquire DoubleClick for $3.1 billion, transferring to Google valuable relationships that DoubleClick had with Web publishers and advertising agencies.",

"Sheikh Mohamed bin Zayed Al Nahyan GCMG was born on the 11th of March 1961.",

"Ouagadougou is the capital of Burkina Faso and the administrative, communications, cultural, and economic centre of the nation.",

"President Volodymyr Zelensky had a conversation with the president of the United Arab Emirates and thanked him for supporting the sovereignty and territorial integrity of Ukraine."]

ignore last image

a) For each of the following questions, identify the answer type (i.e. In terms of a named entity tag) and the features of the question a system might use to predict these types. W) 'Who is the president of the UAE?' ( 2 marks) ) "Which company did Google buy?" ( 2 marks) i) "Where is Ouagadougou?" (2 marks) (x) b) By following the steps below, build an inverted index for the sentences above. This will be a dictionary whose keys are tokens and whiose values are indexes into the list of sentences. 1) Tokenize the sentences and call the result tokenized_sentences. In other words, produce a list of lists of tokens, by tuming each string in sentences into a list of tokens. For example: ["This is an exanple", "This is another"] would become [["This", "is"," "an", "example"], ["This", "is"," another"]] (4 marks) d) 9 Describe at least two other forms of pre-processing that might be applied to the sentences above, and discuss the impact they might have on the question answering task. ( 5 marks) Double-click (or enter) to edit ii) Explain how a Knowledge Based approach to Question Answering works and how it differs from the Information retrieval approach discussed so far. Give examples of systems that use a knowledge based approach. (10 marks) Double-click (or enter) to edit a) For each of the following questions, identify the answer type (i.e. In terms of a named entity tag) and the features of the question a system might use to predict these types. W) 'Who is the president of the UAE?" ( 2 marks) ) "Which company did Google buy?" ( 2 marks) 1) "Where is Ouagadougou?" (2 marks)