EE 2810, HW 3: Submit either one pdf file (preferred choice) or one word-document for this HW.

Gradient descent is a numerical method for finding the minimum of a function y=f(x). In this HW we are going to write a function that performs the gradient descent method on y=f(x). Then we are going to see the method being applied to y=x² by calling the function we wrote.

1. Open a script and write the following code in the script.

a. Define a variable x to be an array of numbers from -10 to 10 with step sizes of 0.1.

b. Define the array y to be y=x², and plot y as a function of x.

c. From the plot created in (1.b), answer the following questions:

i) Is y a convex function of x?

ii) Does y have a minimum over the range of x? If so, what is the minimum value of y and at what value of x does that minimum occur? What is the gradient of y=x² at that minimum point?

d. Save your script under the name myExample.m

2. Open a new script to write a function called myGradientDescent that takes x and y as an input and gives min_y and x_at_min_y as an output.

3. Now we want to implement the gradient descent method in the function myGradientDescent. The gradient descent method starts at an initial point x0 and keeps moving towards the minimum point with a step size of a until it converges to the minimum. To move towards the minimum point means to move in the direction that reduces the gradient. To converge to the minimum point means that you reach a point where the gradient at that point converges to zero, i.e., the gradient at that point is within a distance of e from zero.

a. Define x0=10, a=0.1, e=0.01 and the condition cnd=1.

b. Calculate the gradient of y at x0 as grad_y_at_x0 = (y_at_x0 y_at_x0_minus_a)/a where y_at_x0 is the value of y at the point x0 and y_at_x0_minus_a is the value of y at the point x0 - a.

c. Write a while loop which keeps executing until cnd=0. In the loop:

i) The value stored in x0 is updated using x0 = x0 (grad_y_at_x0 * a)

ii)The gradient of y is recalculated at the new point x0 using grad_y_at_x0 = (y_at_x0 y_at_x0_minus_a)/a.

iii) The value of the condition cnd is updated using cnd= grad_y_at_x0

d. When the while loop is done executing, the variable x0 would contain the point where the minimum occurred and the variable y_at_x0 will contain the minimum value of y. Now, outside the while loop set min_y=y_at_x0 and x_at_min_y=x0.

4. Open your script myExample.m and continue coding in there:

a. Call the function myGradientDescent using the input x and y and let the output of your function call appear on the command window by not placing a semi-colon at the end of your function call.

i) Using hold on, plot the output of the function call as a star (i.e., *) on top of the plot created in part 1.b. Is the * placed at the minimum of y=x² where you expect it to be? If not, you have a mistake in your code and you need to debug your code!