$3$ IMPLEMENT LOGISTIC REGRESSION USING MATLAB

Create a $10004$ Design Matrix by making a column of $1\text{'}$s $($in the first column$)$ followed by the

three columns of $(x,y,z)$ data, you can call this $x_{?}$full.

There is no need to shuffle this dataset, since it was generated using a random approach.

Separate the design matrix, $x_{?}$full, and desired vector, $Y,$ into training and testing subsets:

x$\_$train, y$\_$train X$\_$test, y$\_$test

You will implement gradient descent to find the optimal weights, w $(1),$ w $(2),$ w $(3),$ w $(4).$

As you know, gradient descent involves the iteration:

vec$(w)(n+1)=$vec$(w)(n)-longra{d}_w(L)$

In this case $L=e^2,$ and $e=y-$hat$(y)=y-(vec(w{)}^t(vec(x))).$ To find the gradient, you'll need to apply the

chain rule a few times. Note that $\frac{d}{dq}(q)=(q)[1-(q)].$

You will find it convenient to define the functions:

$\%$ Compute the sigmoid of a scalar or vector

function $y=$sigmoid$(x)$

$\%$ Compute the derivative of the sigmoid, for a scalar, $x$

function $y=$ dsigmoid $(x)$

even though these are both fairly simple functions. It is very easy for your code complexity to

get out of hand.a$)$ You should initialize your gradient descent by picking random values for the weight vector,

vec$(w).$

b$)$ You will iterate through some number, NUM$\_$EPOCHS, of epochs. You can try

NUM$\_$EPOCHS $=200.$

c$)$ Each epoch will involve iterating through each row of $x$ train. For each row compute

hat$(y)=(vec(w{)}^t(vec(x))),L=e^2,$ and $e=y-$hat$(y)=y-(vec(w{)}^t(vec(x)))$

and then apply the gradient update. You can start with $=0\mathrm{.}1,$ and adjust up or down as

you see fit.

At the end of each epoch, you can easily compute the vector of training and testing errors,

and training and testing loss, e$.$g$.$ for training:

vec$(e{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}=$vec$(y{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}-$vec$(y{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}=$vec$(y{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}-(x_{\text{t}\text{r}\text{a}\text{i}\text{n}}(vec(w))),L_{\text{t}\text{r}\text{a}\text{i}\text{n}}=\frac{\text{v}\text{e}\text{c}(e{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}^t\text{v}\text{e}\text{c}(e{)}_{\text{t}\text{r}\text{a}\text{i}\text{n}}}{N_{\text{t}\text{r}\text{a}\text{i}\text{n}}}$

where $(vec(q))$ produces a vector valued result of a vector valued argument

Record the training and testing loss for each epoch, and plot that during or after training.

The training lost must decrease monotonically $($i$.$e$.$ without ever increasing$).$ If it does not,

you have a bug in your code.

Scatterplot your data, along with your separating hyperplane. Your result should look

something like this:

Here$$s some plotting code to help:

figure$(100)$

$[$x$\_$grd y$\_$grd$]=$ meshgrid$(-3$:$0\mathrm{.}1$:$3)$; $\%$ Generate x and y data for the plane

z $=$ zeros$($size$($x$\_$grd$,1))$; $\%$ Generate z data for the plane

for row $=1$:size$($x$\_$grd$,1)$

for col $=1$:size$($x$\_$grd$,1)$

z$($row$,$col$)=-($w$(1)+$ w$(2)*$x$\_$grd$($row$,$col$)+$ w$(3)*$y$\_$grd$($row$,$col$))/$w$(4)$;

end

end

$\%$ Plot random data points with desired classes

scatter$3($X$\_$train$($:$,2),$ X$\_$train $($:$,3),$ X$\_$train $($:$,4),16,...$

$[$y zeros$($length$($y$),1)(1-$y$)],$ "filled"$)$

hold

surf$($x$\_$grd$,$ y$\_$grd$,$ z$)\%$ Plot the plane, if perfect training should

$\%$ separate the classes, may not be $100\%------$ this is what i have thus far : $\%$ PART $2$: GENERATE TWO$-$CLASS PSEUDO$-$RANDOM DATA IN $3$ DIMENSIONS

$\%$ Assuming student ID digits are already defined as dig$1,$ dig$2,$ dig$3,$ dig$4$

dig$1=5$; $\%$ Example digit

dig$2=9$; $\%$ Example digit

dig$3=2$; $\%$ Example digit

dig$4=7$; $\%$ Example digit must not be $0$

$\%$ Generate the u vector as per instructions

u $=0\mathrm{.}33*[$dig$1$ dig$2$ dig$3$ dig$4]$;

$\%$ Initialize matrices to store data points and classes

arr $=$ zeros$(1000,3)$;

y $=$ zeros$(1000,1)$;

$\%$ Generate $1000$ data points

for i $=1$:$1000$

$\%$ Generate x and y randomly between $0$ and $1$

x $=$ rand$()$;

y$\_$point $=$ rand$()$;

$\%$ Solve for z using the plane equation

z $=-($u$(1)+$ u$(2)*$x $+$ u$(3)*$y$\_$point$)/$ u$(4)$;

$\%$ Generate a random deviation

deviation $=$ rand$()-0\mathrm{.}5$;

$\%$ Find the final coordinates of the point

arr$($i$,$ :) $=[$x y$\_$point z$]+$ deviation $*$ u$(2$:$4)$;

$\%$ Assign class based on the deviation

y$($i$)=$ deviation $>0$;

end

$\%$ PART $3$: IMPLEMENT LOGISTIC REGRESSION USING MATLAB

$\%$ Create the design matrix

X$\_$full $=[$ones$(1000,1),$ arr$]$;