Answered step by step
Verified Expert Solution
Link Copied!

Question

1 Approved Answer

2. For a > 0, a smooth function f : R )- R is a-conoez'c if for all 3:, y E R, M) 2 ms)

image text in transcribed
image text in transcribed
2. For a > 0, a smooth function f : R\" )- R is a-conoez'c if for all 3:, y E R", M) 2 ms) + + Elly 3H5. In this exercise we will show a better bound on convergence rate of gradient descent (GD) for such functions. Let f : R\" ) R be a cr-convex iii-smooth function. Let 33'\" be an optimal minimizer for j. Consider the GD algorithm with starting point In. and step-size t = 1/3. Show that after it iterations we have 59 05 Hits: :r:''||% g \"In M3 (1 3) - In other words, the distance of the k'th iterate to the optimal decreases exponentially in the number of iterations. [4 points] [Hint: You may use the fact that Vf(3:"') = 0.] Remark: Note that the number of iterations to get error a for such functions is O(log(1/s)) for fixed 339,0, .3; this is exponentially better than the bound for general covex functions which was 0(1/5)

Step by Step Solution

There are 3 Steps involved in it

Step: 1

blur-text-image

Get Instant Access to Expert-Tailored Solutions

See step-by-step solutions with expert insights and AI powered tools for academic success

Step: 2

blur-text-image

Step: 3

blur-text-image

Ace Your Homework with AI

Get the answers you need in no time with our AI-driven, step-by-step assistance

Get Started

Recommended Textbook for

Elementary Statistics

Authors: Mario F. Triola

3rd Canadian Edition

032122597X, 978-0321225979

More Books

Students also viewed these Mathematics questions