We are using the following dataset(there are total of 4.4M tweets in this file, but you only need to read 1M): http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt

Write a python coded that will download the adove link's data and save to a local text file. ONLY needs to read 1,000,000 lines worth of tweets.

We cant use: read() or readlines() without any parameters at any point. That command will attempt to read the entire file which is too much data.

This is want I have so far:

import time import urllib.request

Start = time.time()

webFD=urllib.request.urlopen("http://rasinsrv07.cstcis.cti.depaul.edu/CSC455/OneDayOfTweets.txt")

New_file = open('downloadedfile.txt','w')

Line = webFD.readline() - I can use that so what do I need to do instead

tweetsdata= (Line.decode('utf8')).split('EndOfTweet') for L in range(100000): New_file.write(L +' ')

New_file.close()

End=time.time()

print("difference is", (End-Start), "seconds") print("performance:", 100000/(End-Start), "operation per second") print("new file for tweetdata is on the desk top")