Suppose we have three positive examples $x_1=(1,0,0),x_2=(0,0,1)$ and $x_3=(0,1,0)$ and three negative examples $x_4=(-1,0,0),x_5=(0,-1,0)$ and $x_6=(0,0,-1)$. Apply standard gradient ascent method to train a logistic regression classifier (without regularization terms).

Initialize the weight vector with two different values and set $w_0^0=0$ (e.g. $w_0=(0,0,0,0)\prime ,w_0=$ $(0,0,1,0{)}^{\prime })$. Would the final weight vector $\left(w^{\ast }\right)$ be the same for the two different initial values? What are the values? Please explain your answer in detail. You may assume the learning rate to be a positive real constant $\eta$.

Transcribed Image Text:

Tuple # Class Probability 1 P N P P N 23456780 8 9 10 P N N N P 0.95 0.85 0.78 0.66 0.60 0.55 0.53 0.52 0.51 0.40