THE QUESTIONS I HAVE ARE MARKED WITH AN ARROW IN THREE SPOTS. PLEASE HELP!

Logistic Regression with Gradient Descent: A Classification Exercise - #2

In this exercise you are going to complete the code necessary for a supervised classification problem

We start by generating a set of points in 2-dimensional plane. Then we decide on cuve that seperates points into two sets: In this case the inisde and outside of a unit circle. We label the data into variable y for the points inside and outside. Then we pick 5 features that includes the second powers of each variable. Our objective is to use the standard gradient descent method minimize a cost function for the logistic regression hypothesis function, and have the program figure out the optimal coefficients for the correct boundary.

This exercise takes you through 4 basic steps:

Generating a simulated set of data and labels

Constructing a set of features

Wriitng the Cost function for logistic regression

Gradient Descent for logistic regression

You will also apply the packaged logistic regression model from sklearn and compare with our solution

Packages Used:

pandas for data frames and easy to read csv files numpy for array and matrix mathematics functions matplotlib for plotting

In [1]:

import numpy as np import pandas as pd import matplotlib.pyplot as plt

%matplotlib inline

from sklearn.preprocessing import MinMaxScaler #we may need this if the sqaure is larger

np.random.seed(42) m = 5000

#Now generate 5,000 points points in the place within square [-3, 3] X [-3, 3] X_data = np.random.uniform([-3,-3],[3,3], size=(m,2))

......... AFTER GOING THROUGH THE SCATTER PLOT THE NEXT PART FOR ME IS...

Now you going to start coding to develop the logistic regression model that seperates this data. That is, identify the boundary curve

In [15]:

#The function is the basic sigmoid function: You can compelete in a single line of code 

def sigmoid(scores): 

 ------->######## insert your code here ~ one line 

 logits = None 

 return logits 

 ###### 

In [18]:

##Now you will have to complete the code for costFunction: 3 -4 lines of code 

def costFunction(features,labels,weights): 

 m = len(features) 

 --------------> ##### insert your code here ~ three lines 

 logits = None 

 y_hat = None 

 loss_y_hat = None 

 ############ 

 cost = -loss_y_hat/m 

 return cost 

In [19]:

#This is the most interesting part of the assignment: The gradient descent 

# to calculate the optimal parameters 

def log_reg(features, labels, num_steps, learning_rate): 

 weights = np.zeros(features.shape[1]) 

 m = features.shape[0] 

 for step in range(num_steps): 

 scores = np.dot(features, weights) 

 predictions = sigmoid(scores) 

 # Update weights with gradient 

 -------------> ####### insert your code here ~ three lines 

 err = None 

 gradient = None 

 weights = None 

 ####### 

 # Print log-likelihood every so often 

 if step % 10000 == 0: 

 print(costFunction(features,labels,weights)) 

 return weights 

In [20]:

##Let us now use our function to come up with the optimal parameters 

Theta_hat = log_reg(X, labels,num_steps = 30000, learning_rate = 0.01) 

You would see nambers like these:

0.6812963202710484 0.06736286320040136 0.05195177701142407