Classifying Ads Submitted Online Consider the case of a website that caters to the needs of a specific farming community, and carries classified ads intended for that community. Anyone, including robots, can post an ad via a web interface, and the site owners have problems with ads that are fraudulent, spam, or simply not relevant to the community. They have provided a file with 4143 ads, each ad in a row, and each ad labeled as either -1 (not relevant) or 1 (relevant). The goal is to develop a predictive model that can classify ads automatically.

Open the file farm-ads.csv, and briefly review some of the relevant and non-relevant ads to get a flavor for their contents.

Following the class example, please create the TF-IDF feature matrix.

a. Examine the TF-IDF matrix.

i. Is it sparse matrix or dense matrix? (Or answer How much percentage of the entries in the matrix is zero?)

ii. Find two non-zero entries and briefly interpret their meaning, in words. (you do not need to derive their calculation)

b. Using logistic regression, partition the data (60% training, 40% validation), and develop a model to classify the documents as 'relevant' or 'non-relevant.' Comment on your model's accuracy.

Sample Data:

1, ad-jerry ad-bruckheimer ad-chase ad-premier ad-sept ad-th ad-clip ad-bruckheimer ad-chase page found

-1, ad-rheumatoid ad-arthritis ad-expert ad-tip ad-info ad-article ad-treatment ad-option ad-support title-understand title-rheumatoid title-arthritis title-everyday title-health header-understand header-rheumatoid header-arthritis understand rheumatoid arthritis everyday health root root act consumer root content everyday solution understand rheumatoid arthritis future ra treatment advance rheumatoid arthritis treatment expect future lead researcher ra treatment research exercise ra check tip slideshow help create workout program ra fitness tip question doctor print list rheumatoid arthritis question doctor visit list ra question understand rheumatoid arthritis tip manage rheumatoid arthritis pain mak key change help manage rheumatoid arthritis pain ease joint pain strive eat balance diet help healthy weight sufficient vitamin mineral counter chronic inflammation tip ease rheumatoid arthritis pain cause rheumatoid arthritis inflammation rheumatoid arthritis ra symptom cause inflammation learn inflammation lead ra symptom joint pain stiffness plus discover cause rheumatoid arthritis rheumatoid arthritis inflammation rheumatoid arthritis management research rheumatoid arthritis research lifestyle choice day impact ra symptom example people ra experience depression treate depression help people manage ra control ra rheumatoid arthritis expert yoga ra yoga safe exercise option person rheumatoid arthritis read dr susan lee answer root act consumer rheumatoid arthritis poll lifestyle change help manage ra pain please select option eat healthy balance diet muscle strengthen aerobic exercise sleep night try reduce manage stress technique haven lifestyle change toolkit healthy recipe shop list meal planner recipe box tool root act consumer enter search term register sign newsletter home health common condition add adhd addiction allergy alternative health alzheimer disease anxiety disorder arthritis asthma autism autoimmune disorder bipolar disorder pain breast cancer cancer cardiovascular health cold flu dental health depression diabete diet nutrition digestive health dvt emotional health epilepsy erectile dysfunction family health fibromyalgia fitness gerd headache migraine healthy home healthy live heart health cholesterol hiv aid hypertension ib incontinence kid health health menopause multiple sclerosis osteoporosis pain management pet health psoriasis rheumatoid arthritis schizophrenia senior health sexual health skin beauty sleep stop smok stroke swine flu weight women health yeast infection condition drug symptom checker flu checkup abdominal pain arm pain pain body ach breast pain breathing difficulty chest pain congestion cough diarrhea ear pain excessive sweate faintness fatigue fever ga headache irregular period joint pain leg pain mouth lesion nausea neck pain rash rectal bleed skin lump sore throat vaginal itch vomite food fitness calorie counter healthy recipe search recipe diet nutrition weight fitness community profile blog discussion photo albums everyday health health tool bmi calculator bmr calculator body fat calculator brain game conversion calculator glossary glucose tracker meal planner calorie counter photo gallery recipe box symptom checker video weight tracker everyday health edit profile inbox discussion blog friend tool copyright everyday health inc everydayhealth com everyday health inc help ad policy advertise link feedback advertise notice site third party advertisement site collect information visit site website provide advertisement service obtain information advertise practice choice online behavioral advertise please click material web site provide educational purpose medical advice diagnosis treatment additional information site subject term privacy policy site comply honcode standard trustworthy health information verify

-1, ad-rheumatologist ad-anju ad-varghese ad-yonker ad-ny ad-pomona ad-ny ad-www ad-arthritisandrheumatologyny ad-com title-practice title-location practice location arthritis rheumatology pllc rheumatology specialist anju varghese board certify internal medicine rheumatology practice limit rheumatology subtitle text home practice location patient resource rockland county ny medical park dr lower level pomona ny phone fax route exit palisad parkway westchester county ny north broadway nd floor yonker ny phone fax st john riverside hospital exit sawmill river parkway accept patient participate health insurance health plan content copyright host exchange

-1, ad-siemen ad-water ad-remediation ad-water ad-scarce ad-resource ad-siemen ad-help ad-preserve title-siemen title-usa siemen usa skip content siemen skip site identifier siemen usa close site id layer skip language selection skip generic navigation contact skip search search industry energy healthcare business product industry solution motor drive build technology industry automation financial solution solution service lighte osram sylvania product lifecycle management mobility water technology power generation power transmission power distribution automation control protection electrical compression expansion ventilation mechanical drive service financial solution solution service diagnostic image therapy hear aid product laboratory diagnostics build technology financial solution solution service consumer product corporate research government solution information communication siemen financial solution solution service siemen usa investor relation press job career business siemen global website answer america renewable energy smart grid technology medical image electronic healthcare record green build commuter rail system employee siemen unit commit answer america toughest question close productfinder layer close logo layer siemen corporation corporate information privacy policy term digital id

-1, ad-symptom ad-muscle ad-weakness ad-genetic ad-disease ad-symptom ad-include ad-search ad-learn page found

1, ad-animal ad-animal ad-wild ad-sa ad-official ad-tourism ad-website page found

-1, ad-dr ad-enrico ad-fazzini ad-parkinson ad-disease ad-specialist ad-nyu ad-movement ad-disorder ad-neurologist ad-www ad-theparkinsonsdoctor ad-com page found

-1, ad-ulcerative ad-colitis ad-uc ad-quiz ad-ulcerative ad-colitis title-colitis title-treatment title-endless title-path header-colitis header-treatment header-endless header-path colitis treatment endless path colitis treatment endless path inflammatory bowel disease patient complementary treatment fed colitis control life activity read serovera natural potent anti inflammatory proven heal ulcerative colitis people serovera testify significant health improvement ulcerative colitis week feel uc relief day confident serovera effectiveness colitis ll prove unprecedent offer serovera aloe vera powerful anti inflammatory antimicrobial property direction dr ivan danhof ph isolate process heal agent aloe vera allow infuse purest potent medicinal value serovera amp capsule formulate colitis sufferer amp aloe mucilaginous polysaccharide sooth heal bowel inflammation detoxify stomach digestive organ heal intestinal wall damage rebalance cell regenerate tissue stimulate stomach production pepsin digestion enzyme strengthen immune system restore intestinal health help reduce likeliness future flare help remission serovera serovera amp option avoid reason serovera refuse provide feel beware amp brand dilute improperly process quality amp matter quantity thank proprietary freeze dry process amp product compare potency effectiveness serovera unlike brand offer absorption agent option glutamine soy lecithin save money paypass club yield discount reward free bottle serolife probiotics transaction combination serovera serolife probiotics help restore level beneficial bacteria naturally occur probiotics digestive tract pave create stronger bind receptor amp deliver serovera hear satisfy customer ulcerative colitis doctor prescribe medication ineffective serovera little prayer sceptical lack online feedback regard amp week tak virtually symptom beg disease amp shot collegeguy serovera ulcerative colitis ulcerative colitis disease cause inflammation sor call ulcer line rectum colon ulcer form inflammation kill cell usually line colon bleed produce pus inflammation colon cause colon empty frequently cause diarrhea inflammation occur rectum lower colon call ulcerative proctitis entire colon affect call pancolitis left colon affect call limit distal colitis symptom colitis clinical presentation ulcerative colitis depend extent disease process patient usually diarrhea mix blood mucus gradual onset sign weight loss blood rectal examination usually pain associate disease patient describe discomfort pain associate exmple crohn disease ulcerative colitis systemic disease affect body sometime extra intestinal manifestation disease initial sign painful arthritic knee teenager unlikely disease correctly diagnose onset intestinal manifestation video testimonial representative stand ready assist toll free unit kingdom australia international feature

-1, ad-wellcentive ad-registry ad-web ad-base ad-preventive ad-care ad-chronic ad-disease ad-management ad-system ad-www ad-wellcentive ad-com title-wellcentive title-patient title-registry title-system title-instanceendeditable title-instancebegineditable title-name title-head wellcentive patient registry system instanceendeditable instancebegineditable name head wellcentive industry lead preventive care chronic disease management health information exchange system provide significant return investment increase reimbursement pay performance patient center medical home cm pqri arra meaningful program wellcentive registry solution low cost ehr focus actionable patient data instead entire data set emr patient registry system adversely affect office workflow productivity emr reduce barrier encourage adoption wellcentive solution thousand physician database contain million patient record system send receive million administrative clinical patient data message week effectiveness software solution unique approach integration interfac extraordinary focus customer service otherwise unheard pric model help position wellcentive recognize leader quickly grow field emphasis system integration custom interfac mak registry solution community solution valuable addition organization overall hit strategy physician physician organization health information exchange regional extension center employer health plan hit vendor hospital system wellcentive inc wellcentive registry wellcentive ehr wellcentive outreach wellcentive pqri wellcentive enterprise wellcentive gateway wellcentive connect wellcentive support pric faq contact

1, ad-free ad-raw ad-food ad-video ad-feature ad-mike ad-adam ad-david ad-wolfe ad-morgan ad-spurlock ad-tony ad-robbin title-tony title-robbin header-inside header-ll header-discover tony robbin secret surprisingly simple shockingly powerful amazingly easy implement inside ll discover tony deepest health fear live insight change life forever insight stay healthy human help healthy drive unhealthy ah ha moment change life forever specific food tony mak diet easily stay healthy control emotion re driven unhealthy comfort food easier simple easy understand view human psychology time easier eat healthier food resist unhealthy food lot pressure stress eat healthy drug company advertisement stay hook product ll look drug advertisement light learn practical easy implement strategy immediately start eate ll literally able start minute enter email instant access double check email accuracy ensure receive access information privacy email safe secure affiliate privacy policy support term service raw thirty blog copyright rawfor day com

-1, ad-north ad-shore ad-lij ad-cancer ad-center ad-provide ad-art ad-care title-north title-shore title-island title-jewish title-health title-system header-cancer header-treatment north shore island jewish health system cancer treatment largest cancer center york metropolitan north shore lij innovative method forefront cancer care learn north shore lij advancement cancer treatment continuum care physician referral personal reflection patient receive cancer care north shore lij ad benefit access vast resource entire health system diagnostics treatment rehabilitation home care relate medical service offer continuum care learn receive personalize referral cancer specialist cal search online directory hear person story treatment recovery north shore lij learn visit site learn program service copyright north shore lij health system root root

1, ad-world ad-finest ad-alpaca ad-breeder ad-seller ad-finest ad-accoyo ad-peruvian ad-alpaca ad-sunsethillsalpaca ad-com title-champion title-alpaca title-breed title-sale title-sunset title-hill title-farm title-alpaca header-welcome header-sunset header-hill header-farm header-alpaca champion alpaca breed sale sunset hill farm alpaca home alpaca sale herdsir service alpaca investment info record farm degree road butler pa fax info sunsethillsalpaca com welcome sunset hill farm alpaca visit national alpaca farm day saturday september fun fill family day farm meet alpaca hay ride learn alpaca alpaca seminar information establish dr david laurye feller sunset hill farm alpaca commit breed finest quality accoyo peruvian alpaca careful selection top quality alpaca genetics world establish sunset hill farm nation leader award win alpaca multiple color champion blue ribbon herdsir peruvian zephyrus color champion sunset hill jedi polare accoyo kilyo star hill sunset hill andean gold snowmass legend maker ngg lional sunset hill lion king snowmass maharajah jedi accoyo usa tr cuarte accoyo black expresso sunset hill tempest sunset hill farm alpaca offer unique genetic diversity premier bloodline central location pride ourself ongo quality service attention customer call love visit visit alpca de moda fashion life alpaca owner breeder association pennsylvania alpaca owner breeder association celebrity sale futurity alpaca sale breakfast sunset hill farm top page copyright sunset hill farm alpaca llc reserve

1, ad-vet ad-online ad-veterinarian ad-online ad-question ad-answer ad-asap title-pet title-veterinary title-question title-health title-care title-information title-online title-vet title-answer title-dog title-cat title-bird title-fish title-hamster title-gerbil title-ferret title-reptile title-horse title-rabbit title-farm title-animal title-servername title-www header-veterinary header-question header-answer header-asap pet veterinary question health care information online vet answer dog cat bird fish hamster gerbil ferret reptile horse rabbit farm animal servername www register login help veterinary veterinary question answer asap watch video pet veterinary category legal tax legal family law immigration law employment law criminal law military law real estate law canada law personal injury law business law consumer protection law estate law bankruptcy law australia law intellectual property law south africa law zealand law republic ireland law uk law uk family law uk property law uk immigration law scot law uk employment law uk bankruptcy law uk traffic law tax finance uk tax financial software car vehicle car bmw motorcycle merce vw volvo audi classic car jaguar subaru rv australia car kia porsche hyundai mitsubishi mazda saab boat marine electronics chrysler dodge jeep ford mercury lincoln gmc chevy buick pontiac cadillac saturn honda acura toyota lexus uk car uk ford uk nissan nissan infiniti heavy equipment health medical health dental mental health pharmacy medical ob gyn pediatrics urology eye dermatology oncology neurology bariatrics plastic surgery pet veterinary pet dog cat bird reptile horse management veterinary dog veterinary cat veterinary bird veterinary horse veterinary animal veterinary home appliance home improvement appliance electrical engine plumb uk appliance hvac uk plumb appliance pool spa electronics tv camera video car electronics printer office exercise equipment cell phone electronic musical instrument computer education computer network mac program homework math french spanish german write business life personal relationship parent entertainment wed job firearm enter question subcategory select expert character left veterinarian online legal tax car vehicle health medical pet veterinary home appliance computer education life personal meet expert dr jodi lynn smith veterinarian positive feedback accept dvm clinical surgical experience animal animal exotics http ww justanswer com upload jodilynnsaleo june jodi jpg dr jodi lynn smith avatar dr jodi lynn smith veterinarian positive feedback accept dvm clinical surgical experience animal animal exotics http ww justanswer com upload isudvm phpnknjmr pm jpg dr bruce avatar dr bruce veterinarian positive feedback accept practical experience er day practice http ww justanswer com upload lpott loretta portrait jpg dr loretta avatar dr loretta veterinarian positive feedback accept experience animal medicine http ww justanswer com upload southernbadger brett jpg dr christian avatar dr christian veterinarian positive feedback accept veterinary experience surgery medicine behavior http ww justanswer com upload vetforyou justanswer photo jpg dr andy avatar dr andy veterinarian positive feedback accept uc dave graduate internal medicine dermatology pain control emergency geriatrics http ww justanswer com upload doggfone vet pic jpg dr scott avatar dr scott veterinarian positive feedback accept ten animal equine pocket pet experience medicine surgery http ww justanswer com upload pcbobber doc jpg dr bob avatar dr bob veterinarian positive feedback accept practice customer professional personally concern answer question george happy found justanswer google search favorite list love kitty mak life complete bev george boca raton fl professional personally concern answer question george happy found justanswer google search favorite list love kitty mak life complete bev george boca raton fl outstand response time minute answer question professionally deal compassion kevin beaverton please speed advice receive add try advice time continue jan spokane wa dollar hour time drove diagnosis mark thank phil marysville ca lurch thank real doubt website promptness response quick followup answer picture incredible charle walnut creek ca wonderful service prompt efficient accurate couldn thank help mary freshfield liverpool uk expert wonderful truly talk actually care help nerve ease thank alex lo angele ca learn justanswer doc web site ve quick question try answer site various specialist hand quick answer justanswer com justanswer com seen spike october legal question reader layoff unemployment severance web site justanswer com legal leave chance traffic justanswer rose percent nearly page view day inquiry relate stress blood pressure drink heart pain jump percent retire nypd mechanic sign expert justanswer com san francisco base advice site answer five eight online question day issue rang technical check engine light help protect furry friend tap fiona fisher veterinary expert justanswer com roundup common household food item pose threat belove pet avoid common toxin dog cat live healthy live recent question alway extremely cat declaw veterinarian answer plus bonus stop month female cat stop bite veterinarian answer short haire male cat veterinarian answer question justanswer question list register login tell friend expert help career term service privacy security affiliate expert referral create category category justanswer corp health dental mental health pharmacy medical ob gyn pediatrics urology eye dermatology oncology neurology bariatrics plastic surgery legal family law immigration law employment law criminal law military law real estate law canada law personal injury law business law consumer protection law estate law bankruptcy law australia law intellectual property law south africa law zealand law republic ireland law uk law uk family law uk property law uk immigration law scot law uk employment law uk bankruptcy law uk traffic law car bmw motorcycle merce vw volvo audi classic car jaguar subaru rv australia car kia porsche hyundai mitsubishi mazda saab chrysler dodge jeep gmc chevy buick pontiac cadillac saturn ford mercury lincoln honda acura toyota lexus nissan infiniti uk car uk ford uk nissan heavy equipment veterinary dog veterinary cat veterinary bird veterinary horse veterinary animal veterinary pet dog cat bird reptile horse management tax finance uk tax financial software computer network mac program homework math french spanish german write business home improvement appliance electrical engine plumb uk appliance hvac uk plumb appliance pool spa electronics tv camera video car electronics printer office exercise equipment cell phone electronic musical instrument boat marine electronics relationship parent entertainment wed job firearm root