Answered step by step
Verified Expert Solution
Question
1 Approved Answer
In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from
In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from PyTorch other than the ones that help download the data. Use Numpy to code up the optimization algorithm. (a) (0 points) First, prepare the dataset. Select all the samples belonging to the first two classes from MNIST's training dataset, this will be your training dataset. Similarly, create a validation dataset for the two classes by picking samples corresponding to the first two classes from the validation set of the MNIST dataset. You can subsample input images from 2828 to 1414 if you need. (b) (10 points) Logistic regression solves for 1 n (1+ Yi argmin Clog (1 + eyi (wxi+wo)) n wRd,woR i=1 + (|||| + + w%2) (1) where x = R196. Set y = 1 for MNIST images labeled zero and y; = -1 for MNIST images labeled one. Initialize the weights randomly for both the following parts but make sure that they are the same for both gradient descent and gradient descent with Nesterov's acceleration in part (c). You can try a few different values of X and pick the one that gives the best validation error for the following parts. Optimize the objective in (1) using gradient descent (note, not stochastic gradient descent) and plot the training loss as a function of the number of parameter updates on a semi-log scale (log scale on the Y-axis). This plot should be a straight line. As we saw in the class, the slope of this line should be about -- for gradient descent. Compute the slope of the line in your plot and mention it clearly. (c) (5 points) Write down the Hessian of the loss function in (1). Without assuming any special A conditions about the dataset {(xi, Yi)} i=1,...,., is this problem strongly convex? What is the best strong convexity parameter for the loss function in (1)?
Step by Step Solution
★★★★★
3.49 Rating (159 Votes )
There are 3 Steps involved in it
Step: 1
Lets break down the problem into parts and address each one a Prepare the dataset import numpy as np from torchvision import datasets transforms Load ...Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started