Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

I wrote this page scraper using python. It works but the formatting to the csv file is incorrect and need some help to fix it.

I wrote this page scraper using python. It works but the formatting to the csv file is incorrect and need some help to fix it. The tire name is on two lines for most of the tires, and the prices and reviews need to all be shifted up one line. How can I do this? For example the first tire in the csv file, Goodyear Assurance All-season, should all be on one line. Thank you

import requests import csv from datetime import datetime from bs4 import BeautifulSoup

# for tire and review page # compile with just python, not python3

# download the page myurl = requests.get("https://www.goodyearautoservice.com/en-US/tires/all-season")

# create BeautifulSoup object page_soup = BeautifulSoup(myurl.text, 'html.parser')

''' # get all the tire info we need, specify where the info is located h3 = soup.find_all('h3', class_ = "baseball-card__title")

for tireblock in h3: # find the children in tire block children = tireblock.findChildren() first = [] for child in children: # grab the text in the b tags print(child.get_text())

price = soup.find_all('span', class_ = "baseball-card__price") for listprice in price: orig = price[0].text.strip() print(orig) ''' containers = page_soup.findAll("div",{"class":"baseball-card baseball-card__wrapper"})

filename = "product.csv" o = open(filename, "w")

header_row = "Tire, Price, Five Star Reviews "

o.write(header_row)

for container in containers: # product name tire_name = container.findAll("a", {"class":"link-chevron"}) product_name = unicode(tire_name[0].text).encode('utf-8').strip() print(product_name)

# tire price price = container.findAll("span", {"class":"baseball-card__price"}) list_price = unicode(price[0].text).encode('utf-8').strip() print(list_price)

# tire review reviews = container.findAll("span", {"class": "my-store__rating__stars"}) if reviews: tire_reviews = unicode(reviews[0].text).encode('utf-8').strip() print(tire_reviews) else: tire_reviews="No reviews available"

print("Tire: " + product_name) print("Price: " + list_price) print("Five Star Reviews: " + tire_reviews)

o.write(product_name + "," + list_price + "," + tire_reviews + " ")

o.close()

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Machine Learning And Knowledge Discovery In Databases European Conference Ecml Pkdd 2016 Riva Del Garda Italy September 19 23 2016 Proceedings Part 1 Lnai 9851

Authors: Paolo Frasconi ,Niels Landwehr ,Giuseppe Manco ,Jilles Vreeken

1st Edition

3319461273, 978-3319461274

More Books

Students also viewed these Databases questions