Answered step by step
Verified Expert Solution
Question
1 Approved Answer
Consider the strongly convex function y[x] = (x 2). 1. Analytically obtain the global minimizer of the function via taking the gradient of the
Consider the strongly convex function y[x] = (x 2). 1. Analytically obtain the global minimizer of the function via taking the gradient of the function and finding the root of the gradient. 2. Assume that you have written a code for gradient descent over this function and xo=0 is chosen as the initial point. Mimic what the code would do and fill out the following steps for three different choices of step size (n = {0.001, 0.1,5}): Iteration 1: x = xo nVy[xo] X =? Iteration 2: x = x nVy[x] x =? Iteration 2: x3 = x nVy[x] X3 =? Iteration 2: x4 = x3 nVy[x3] X4 =? Iteration 2: *5 = x4 nVy[x4] X5 =? you need to generate 15 numbers (5 iterations for each choice of step size) " 3. What is the difference in terms of the behavior of the solutions (i.e., x, x5) for the three aforementioned choices of step size? Hint: How is the sequence approaching or bouncing back and forth around the global minimizer? Hint: 4. Given the above intuition, what do you think about this statement: "It is always better to choose a smaller step size for gradient descent". Hint: think about the speed of conver- gence vs. accuracy! Sometimes there is no definite Yes/No answer and we have to study the tradeoff.
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started