Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

Add an optional parameter limit with a default of 10 to crawl() function which is the maximum number of web pages to download . Save

Add an optional parameter limit with a default of 10 to crawl() function which is the maximum number of web pages to download . Save files to pages dir using the MD5 hash of the pages URL and Only crawl URLs that are in landmark.edu domain (*.landmark.edu)

Use a regular expression when examining discovered links . import hashlib filename = 'pages/' + hashlib.md5(url.encode()).hexdigest() + '.html' import re p = re.compile('ab*') if p.match('abc'): print("yes")

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image_2

Step: 3

blur-text-image_3

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Big Data 29th British National Conference On Databases Bncod 2013 Oxford Uk July 2013 Proceedings Lncs 7968

Authors: Dan Olteanu ,Georg Gottlob ,Christian Schallhart

2013th Edition

3642394663, 978-3642394669

More Books

Students also viewed these Databases questions

Question

What is a dependent variable?

Answered: 1 week ago

Question

Describe the linkages between HRM and strategy formulation. page 74

Answered: 1 week ago

Question

Identify approaches to improving retention rates.

Answered: 1 week ago