Q$2.$ Logistic Regression: Code $[25]$

In this task, you will learn to build a Logistic Regression Classifier for the same "Financial

Phrasebank" dataset. Bag of Words model will be used for this task.

Use $60\%$ of the data selected randomly for training, $20\%$ selected randomly for testing and

the remaining $20\%$ for validation set. Use classes 'positive' and 'negative' only. Perform

the same cleaning tasks on the text data and build a vocabulary of the words.

Using CountVectorizer ${?}^3,$ fit the cleaned train data. This will create the bag$-$of$-$words

model for the train data. Transform test and validation sets using same CountVectorizer.

To implement the logistic regression using following equations,

$z_i=W*x_i$

hat$(y{)}_i=(z_i)$

we need the weight vector W$.$ Create an array of dimension equal to those of each $x_i$

from the CountVectorizer.

Apply above equations over whole training dataset and calculate hat$(y)$ and cross$-$entropy

loss $L_{CE}$ which can be calculated as

$L_{CE}=-yloghat(y)+(1-y)log(1-$hat$(y))$

Now, update the weights as follows:

$W_{t+1}=W_t-(hat(y{)}_i-y_i)*x_i$

Here, $(hat(y{)}_i-y_i)*x_i$ is the gradient of sigmoid function and $=0\mathrm{.}01$ is the learning rate.

Repeat step $4$ and step $5$ for $500$ iterations or epochs. For each iteration, calculate the

cross$-$entropy loss on validation set.

Calculate the accuracy and macro$-$average precision, recall, and F$1$ score and provide

the confusion matrix on the test set.

Experiment with varying values of $in(0\mathrm{.}0001,0\mathrm{.}001,0\mathrm{.}01,0\mathrm{.}1).$ Report your observations

with respect to the performance of the model. You can vary the number of iterations to

enhance the performance if necessary.