Question
In python, Need help writing loop for a TF-IDF Vectorizer that goes through the iterations of max_features, min_df, max_df and chooses the best parameters for
In python,
Need help writing loop for a TF-IDF Vectorizer that goes through the iterations of max_features, min_df, max_df and chooses the best parameters for logistic regression. The parameters should give the max accuracy, precision, recall, F-1. When sending the data into test and train sets, the random seed is 100, 50% split size. Loop should go through 10 iterations. I have provided the most parts of the loop, and I need help with the syntax and process of the loop.
vectorizer = CountVectorizer(max_features=1601,min_df=3, max_df=1.0, stop_words=stopwords.words('english'))
X = vectorizer.fit_transform(norm_corpus).toarray()
transformer = TfidfTransformer()
X=transformer.fit_transform(X).toarray()
text_train, text_test,sent_train, sent_test = train_test_split (X,y,test_size=0.5, random_state=100)
classifier = LogisticRegression()
classifier.fit(text_train,sent_train)
sent_pred = classifier.predict(text_test)
cm = confusion_matrix(sent_test,sent_pred)
tot=cm[0][0]+cm[1][1]
print (tot)
#Model accuracy
print(tot/cm.sum())
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started