Question
I wrote this page scraper using python. It works but the formatting to the csv file is incorrect and need some help to fix it.
I wrote this page scraper using python. It works but the formatting to the csv file is incorrect and need some help to fix it. The tire name is on two lines for most of the tires, and the prices and reviews need to all be shifted up one line. How can I do this? For example the first tire in the csv file, Goodyear Assurance All-season, should all be on one line. Thank you
import requests import csv from datetime import datetime from bs4 import BeautifulSoup
# for tire and review page # compile with just python, not python3
# download the page myurl = requests.get("https://www.goodyearautoservice.com/en-US/tires/all-season")
# create BeautifulSoup object page_soup = BeautifulSoup(myurl.text, 'html.parser')
''' # get all the tire info we need, specify where the info is located h3 = soup.find_all('h3', class_ = "baseball-card__title")
for tireblock in h3: # find the children in tire block children = tireblock.findChildren() first = [] for child in children: # grab the text in the b tags print(child.get_text())
price = soup.find_all('span', class_ = "baseball-card__price") for listprice in price: orig = price[0].text.strip() print(orig) ''' containers = page_soup.findAll("div",{"class":"baseball-card baseball-card__wrapper"})
filename = "product.csv" o = open(filename, "w")
header_row = "Tire, Price, Five Star Reviews "
o.write(header_row)
for container in containers: # product name tire_name = container.findAll("a", {"class":"link-chevron"}) product_name = unicode(tire_name[0].text).encode('utf-8').strip() print(product_name)
# tire price price = container.findAll("span", {"class":"baseball-card__price"}) list_price = unicode(price[0].text).encode('utf-8').strip() print(list_price)
# tire review reviews = container.findAll("span", {"class": "my-store__rating__stars"}) if reviews: tire_reviews = unicode(reviews[0].text).encode('utf-8').strip() print(tire_reviews) else: tire_reviews="No reviews available"
print("Tire: " + product_name) print("Price: " + list_price) print("Five Star Reviews: " + tire_reviews)
o.write(product_name + "," + list_price + "," + tire_reviews + " ")
o.close()
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started