Question must be done in Python 3.6

It is clever to make your own tweets.txt file with fewer tweets. Processing the entire file takes quite a long time.

We are using a version of TF-IDF that is not completely to the specifications of TF-IDF. Follow the instructions as specified here. With true TF-IDF, the word frequency would be per-document. This makes our output not as valuable as a proper TF-IDF implementation.

You may read the file many times, or save the file in a variable.

Question 2 Write a program that computes the TF-IDF of a file of tweets. TF-IDF is a function that finds highly discriminating words or important words from a set of documents. If the document are argumentive, the important words might by "You" and "T". If the documents are about sports, the important words might be "win", "lose", or "score TF-IDF uses two different functions to calculate what an 'important word entails. The word should not be too frequent, or too infrequent. TF stands for Term Frequency, which is the number of time the word appears in all the documents. IDF stands for Inverse Document Frequency - a function r g the number of documents this word appears in. TF-IDF functions These are the calculations for TF-IDF. Write these into your functions, where appropriate. TF Text frequency is defined by: The number of times the word w appears in all the documents, divided by the number of words in all the documents. freq(w) total unique words TF- IDF # of documents with word w # of documents IDF--log TF-IDF Note that the - is this case is not a subtraction, it's just how it is stylized. It's actually just a mulitplication. TFIDF TF IDF Functions to writ You must have functions that perform the following functions Clean word Write a function that accepts a single word as a parameter, sets the word to lower case, and removes the following punctuation: . ,-)