Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

need this question to be answerd with explinations please The main goal of this homework is to apply the problem solving process to address two

need this question to be answerd with explinations please

The main goal of this homework is to apply the problem solving process to address two practical issues in the real world. You are required to describe your algorithms in pseudo code. You are not required to implement your algorithms.

Submit your two algorithms in pseudo code as one PDF file on iLearn.

Problem 1. identifying most and least frequently co-occurring word-pairs in an article. (20 points)

Given an article (for example, this one at nytimes.com), design an algorithm to find the top 100 most frequently co-occurring word-pairs and the top 20 least frequently co-occurring word-pairs in this article. Two words (i.e., a word-pair) are said to co-occur if they appear in the same sentence. The frequency of a word-pair is defined as the number of sentences that contain this word-pair. When ranking the word-pairs by frequency, if two or more word-pairs have the same frequency, each of them will occupy a separate rank. (It doesn't matter which goes first or second.)

For instance, the last sentence in the above article Its really a milestone in Chinese science fiction. contain the following word pairs:

('it's', 'really')

('it's', 'a')

('it's', 'milestone')

('it's', 'in')

('it's', 'Chinese')

('it's', 'science')

('it's', 'fiction')

('really', 'a')

('really', 'milestone')

('really', 'in')

('really', 'Chinese')

('really', 'science')

('really', 'fiction')

('a', 'milestone')

('a', 'in')

('a', 'Chinese')

('a', 'science')

('a', 'fiction')

('milestone', 'in')

('milestone', 'Chinese')

('milestone', 'science')

('milestone', 'fiction')

('in', 'Chinese')

('in', 'science')

('in', 'fiction')

('Chinese', 'science')

('Chinese', 'fiction')

('science', 'fiction')

Note that the order of two words doesn't matter in a word-pair. That is, the word pair ('science', 'fiction') is the same as ('fiction', 'science'). Also, all the tokens are case-insensitive. Please consider to convert them to all lower-case. In the interest of simplicity, we will only consider co-occurring word pairs that consist of two different words. For instance, we will NOT consider ('the', 'the') as a co-occurring pair in the sentence "This is the best place in the world."

You can assume that you have access to a subroutine, sentenceSplitter(article), that can accurately segment an article into separate sentences and return these sentences in an array-like data structure.

You can also assume that you have access to another routine tokenizer( sentence), that can accurately identify the individual words contained in the input sentence and return these words in another array-like data structure.

Please describe your algorithm unambiguously using pseudo code with necessary comments in English.

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Deductive And Object Oriented Databases Second International Conference Dood 91 Munich Germany December 18 1991 Proceedings Lncs 566

Authors: Claude Delobel ,Michael Kifer ,Yoshifumi Masunaga

1st Edition

3540550151, 978-3540550150

More Books

Students also viewed these Databases questions

Question

Discuss security concerns involved in building a data warehouse.

Answered: 1 week ago

Question

What is happening in this situation? What are the consequences?

Answered: 1 week ago