Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from

In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST.

In this problem, we will implement logistic regression for classifying two classes (zero and one) from MNIST. You may not use any routines from PyTorch other than the ones that help download the data. Use Numpy to code up the optimization algorithm. (a) (0 points) First, prepare the dataset. Select all the samples belonging to the first two classes from MNIST's training dataset, this will be your training dataset. Similarly, create a validation dataset for the two classes by picking samples corresponding to the first two classes from the validation set of the MNIST dataset. You can subsample input images from 2828 to 1414 if you need. (b) (10 points) Logistic regression solves for 1 n (1+ Yi argmin Clog (1 + eyi (wxi+wo)) n wRd,woR i=1 + (|||| + + w%2) (1) where x = R196. Set y = 1 for MNIST images labeled zero and y; = -1 for MNIST images labeled one. Initialize the weights randomly for both the following parts but make sure that they are the same for both gradient descent and gradient descent with Nesterov's acceleration in part (c). You can try a few different values of X and pick the one that gives the best validation error for the following parts. Optimize the objective in (1) using gradient descent (note, not stochastic gradient descent) and plot the training loss as a function of the number of parameter updates on a semi-log scale (log scale on the Y-axis). This plot should be a straight line. As we saw in the class, the slope of this line should be about -- for gradient descent. Compute the slope of the line in your plot and mention it clearly. (c) (5 points) Write down the Hessian of the loss function in (1). Without assuming any special A conditions about the dataset {(xi, Yi)} i=1,...,., is this problem strongly convex? What is the best strong convexity parameter for the loss function in (1)?

Step by Step Solution

3.49 Rating (159 Votes )

There are 3 Steps involved in it

Step: 1

Lets break down the problem into parts and address each one a Prepare the dataset import numpy as np from torchvision import datasets transforms Load ... blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Smith and Roberson Business Law

Authors: Richard A. Mann, Barry S. Roberts

15th Edition

1285141903, 1285141903, 9781285141909, 978-0538473637

More Books

Students also viewed these Programming questions