Answered step by step
Verified Expert Solution
Question
1 Approved Answer
Consider the following training dataset: We would like to fit a linear regression model y^=w0+w1x to this dataset using gradient lescent and 2 regularization, with
Consider the following training dataset: We would like to fit a linear regression model y^=w0+w1x to this dataset using gradient lescent and 2 regularization, with =0.1 and =0.1. We use mean squared error for our oss function and initialize w=(w0,w1)=0 Part(a): Perform the first two gradient updates to w (i.e. what are the weights after one gradient update? after two?). Part(b) : Perform the first two gradient updates to w with momentum using =0.9 (assume z(0)=0 for the first gradient update). Part(c) : What would happen if we set to be much smaller, e.g. =10100 ? What about much larger, e.g. =100 ? (You do not need to perform the computations manually; a written description is sufficient.)
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started