Question
need this question to be answerd with explinations please The main goal of this homework is to apply the problem solving process to address two
need this question to be answerd with explinations please
The main goal of this homework is to apply the problem solving process to address two practical issues in the real world. You are required to describe your algorithms in pseudo code. You are not required to implement your algorithms.
Submit your two algorithms in pseudo code as one PDF file on iLearn.
Problem 1. identifying most and least frequently co-occurring word-pairs in an article. (20 points)
Given an article (for example, this one at nytimes.com), design an algorithm to find the top 100 most frequently co-occurring word-pairs and the top 20 least frequently co-occurring word-pairs in this article. Two words (i.e., a word-pair) are said to co-occur if they appear in the same sentence. The frequency of a word-pair is defined as the number of sentences that contain this word-pair. When ranking the word-pairs by frequency, if two or more word-pairs have the same frequency, each of them will occupy a separate rank. (It doesn't matter which goes first or second.)
For instance, the last sentence in the above article Its really a milestone in Chinese science fiction. contain the following word pairs:
('it's', 'really')
('it's', 'a')
('it's', 'milestone')
('it's', 'in')
('it's', 'Chinese')
('it's', 'science')
('it's', 'fiction')
('really', 'a')
('really', 'milestone')
('really', 'in')
('really', 'Chinese')
('really', 'science')
('really', 'fiction')
('a', 'milestone')
('a', 'in')
('a', 'Chinese')
('a', 'science')
('a', 'fiction')
('milestone', 'in')
('milestone', 'Chinese')
('milestone', 'science')
('milestone', 'fiction')
('in', 'Chinese')
('in', 'science')
('in', 'fiction')
('Chinese', 'science')
('Chinese', 'fiction')
('science', 'fiction')
Note that the order of two words doesn't matter in a word-pair. That is, the word pair ('science', 'fiction') is the same as ('fiction', 'science'). Also, all the tokens are case-insensitive. Please consider to convert them to all lower-case. In the interest of simplicity, we will only consider co-occurring word pairs that consist of two different words. For instance, we will NOT consider ('the', 'the') as a co-occurring pair in the sentence "This is the best place in the world."
You can assume that you have access to a subroutine, sentenceSplitter(article), that can accurately segment an article into separate sentences and return these sentences in an array-like data structure.
You can also assume that you have access to another routine tokenizer( sentence), that can accurately identify the individual words contained in the input sentence and return these words in another array-like data structure.
Please describe your algorithm unambiguously using pseudo code with necessary comments in English.
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started