Answered step by step
Verified Expert Solution
Question
1 Approved Answer
2. For a > 0, a smooth function f : R )- R is a-conoez'c if for all 3:, y E R, M) 2 ms)
2. For a > 0, a smooth function f : R\" )- R is a-conoez'c if for all 3:, y E R", M) 2 ms) + + Elly 3H5. In this exercise we will show a better bound on convergence rate of gradient descent (GD) for such functions. Let f : R\" ) R be a cr-convex iii-smooth function. Let 33'\" be an optimal minimizer for j. Consider the GD algorithm with starting point In. and step-size t = 1/3. Show that after it iterations we have 59 05 Hits: :r:''||% g \"In M3 (1 3) - In other words, the distance of the k'th iterate to the optimal decreases exponentially in the number of iterations. [4 points] [Hint: You may use the fact that Vf(3:"') = 0.] Remark: Note that the number of iterations to get error a for such functions is O(log(1/s)) for fixed 339,0, .3; this is exponentially better than the bound for general covex functions which was 0(1/5)
Step by Step Solution
There are 3 Steps involved in it
Step: 1
Get Instant Access to Expert-Tailored Solutions
See step-by-step solutions with expert insights and AI powered tools for academic success
Step: 2
Step: 3
Ace Your Homework with AI
Get the answers you need in no time with our AI-driven, step-by-step assistance
Get Started