Question
You are required to build a text classifier that can predict whether a given text is relevant or not. The target column is called relevance.
You are required to build a text classifier that can predict whether a given text is relevant or not. The target column is called "relevance". It has 3 values - no (6571), yes (1420), and not sure (9). Turn this into a binary classification problem ("yes" or "no") by deleting rows where relevance = "not sure".
You will have to preprocess the data to get reasonable accuracies. Try the following algorithms: Naive Bayes, Logistic Regression, and Random Forest. Also, use the Lime Text Explainer to identify the words that are most likely to predict relevance (i.e., "yes") vs non-relevance (i.e., "no").
You will do everything in a single IPython notebook. Submit the IPython notebook by the deadline.
Below is the format of the data:
A11 fx U.S. Dollar Falls Against Most Currencies; Decline Is Softened as Bond Rally Stalls B C headline text relevance Yields on CDs Fell in the Latest Week NEW YORK -- Yields on most oertificates of deposit offered by m. yes The Morning Brief: White House Seeks to Limit Child Insurance Program The Wall Street Journal Online The Morning Brief, a look no Banking Bill Negotiators Set Compromise --- Plan to widen Banks' Entry To Securities Business Is Dropped as Vote Nears WASHINGTON -- In an effort to achieve banking reform, Senaten no Manager's Journal: Sniffing Out Drug Abusers Is No Quick Fix The statistics on the enormous costs of employee drug abuse are no Currency Trading: Dollar Remains in Tight Ranges Amid Wait for U.S. Jobs Data NEW YORK -- Indecision marked the dollar's tone, as traders paus yes Stocks Fall Again; BofA, Alcoa Slide Stocks declined, as investors weighed slower-than-expected dom yes Bank of Montreal, Royal Bank Profits Rose in 2nd Period TORONTO -- Royal Bank of Canada and Bank of Montreal poste no Battle Over Medical Costs Isn't Over Many people think that the monster of health-care inflation has be no Sequenom to Buy Gemini Genomics In Stock Accord Sequenom Inc., a genomics-based biotechnology company, said i no U.S. Dollar Falls Against Most Currencies: Decline Is Softened as Bond Rally Stalls The U.S. dollar declined against most major foreign currencies yes yes Dow Falls 45.95, Late GM Surge Stanches Losses The Wall Street Journal OnlineA LATE POP by General no 3 MCI Surpasses Expectations On Earnings MCI World Com Inc., fueled by growth in Internet traffic, data and ol no Defending Yourself Against Deflation Author: James B. Stewart The dreaded "D" word is back yes Lending to Firms Projected to Grow 9.2% Next Year NEW YORK - Commercial and industrial loan volume in the U.S. is no U.S., week ahead of economic summit, signals it wants lower interest rates WASHINGTON -- A week before an international economic sumi no Net Worth, Not Income, Is Right Measure of Inequality According to Kevin A. Hassett and Aparna Mathur'slop-ed, Oct. 2 no The Americas: Ecuador Struggles Toward a Dollar Economy OTAVALO, Ecuador -- The long black braid, the blinding white cott no Dollar Declines as Players Take Profits From Rally and After Fed Boosts Rates NEW YORK -- The dollar sagged yesterday, as news of a widely and yes o In Europe, Job Protections for Older Generation Are Barriers for Younger Workers; Earnings Gap Looms for Younger Generatic By the time the parents of Serena Violano were in their early 30s, th no 1 Outlook for Business Investment Improves, but Softness May Persist The outlook for business investment appears to be improving, bu no 2 Tech Sector in Hiring Drive: Google, Intel Add Workers as Profits Snap Back: Start-Ups Also Fight for Talent The technology industry, an engine of innovation and U.S. prosper yes 3 Fed's Greenspan Refuses to Accept Blame for Recession; Upturn's Pace Is 'Glacial WASHINGTON -- While acknowledging that the pace of economic yes 4 They're Writing the Checks; Ray Lane and Matthew Rogers on the new energy math for venture capitalists Venture capitalists never know for sure which energy innovations no 5 Deal Journal / Breaking Insight From WSJ.com Hard-Hit SharesPut to Good Use--no 3 A11 fx U.S. Dollar Falls Against Most Currencies; Decline Is Softened as Bond Rally Stalls B C headline text relevance Yields on CDs Fell in the Latest Week NEW YORK -- Yields on most oertificates of deposit offered by m. yes The Morning Brief: White House Seeks to Limit Child Insurance Program The Wall Street Journal Online The Morning Brief, a look no Banking Bill Negotiators Set Compromise --- Plan to widen Banks' Entry To Securities Business Is Dropped as Vote Nears WASHINGTON -- In an effort to achieve banking reform, Senaten no Manager's Journal: Sniffing Out Drug Abusers Is No Quick Fix The statistics on the enormous costs of employee drug abuse are no Currency Trading: Dollar Remains in Tight Ranges Amid Wait for U.S. Jobs Data NEW YORK -- Indecision marked the dollar's tone, as traders paus yes Stocks Fall Again; BofA, Alcoa Slide Stocks declined, as investors weighed slower-than-expected dom yes Bank of Montreal, Royal Bank Profits Rose in 2nd Period TORONTO -- Royal Bank of Canada and Bank of Montreal poste no Battle Over Medical Costs Isn't Over Many people think that the monster of health-care inflation has be no Sequenom to Buy Gemini Genomics In Stock Accord Sequenom Inc., a genomics-based biotechnology company, said i no U.S. Dollar Falls Against Most Currencies: Decline Is Softened as Bond Rally Stalls The U.S. dollar declined against most major foreign currencies yes yes Dow Falls 45.95, Late GM Surge Stanches Losses The Wall Street Journal OnlineA LATE POP by General no 3 MCI Surpasses Expectations On Earnings MCI World Com Inc., fueled by growth in Internet traffic, data and ol no Defending Yourself Against Deflation Author: James B. Stewart The dreaded "D" word is back yes Lending to Firms Projected to Grow 9.2% Next Year NEW YORK - Commercial and industrial loan volume in the U.S. is no U.S., week ahead of economic summit, signals it wants lower interest rates WASHINGTON -- A week before an international economic sumi no Net Worth, Not Income, Is Right Measure of Inequality According to Kevin A. Hassett and Aparna Mathur'slop-ed, Oct. 2 no The Americas: Ecuador Struggles Toward a Dollar Economy OTAVALO, Ecuador -- The long black braid, the blinding white cott no Dollar Declines as Players Take Profits From Rally and After Fed Boosts Rates NEW YORK -- The dollar sagged yesterday, as news of a widely and yes o In Europe, Job Protections for Older Generation Are Barriers for Younger Workers; Earnings Gap Looms for Younger Generatic By the time the parents of Serena Violano were in their early 30s, th no 1 Outlook for Business Investment Improves, but Softness May Persist The outlook for business investment appears to be improving, bu no 2 Tech Sector in Hiring Drive: Google, Intel Add Workers as Profits Snap Back: Start-Ups Also Fight for Talent The technology industry, an engine of innovation and U.S. prosper yes 3 Fed's Greenspan Refuses to Accept Blame for Recession; Upturn's Pace Is 'Glacial WASHINGTON -- While acknowledging that the pace of economic yes 4 They're Writing the Checks; Ray Lane and Matthew Rogers on the new energy math for venture capitalists Venture capitalists never know for sure which energy innovations no 5 Deal Journal / Breaking Insight From WSJ.com Hard-Hit SharesPut to Good Use--no 3Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started