The analysis of gradient descent and the choice of step size in every iteration is beyond the scope of this class, but the implementation of this algorithm requires one to compute gradients of the function at various points as given in Step $1$ of the algorithm. Hence, the computational complexity of gradient descent boils down to the complexity of evaluating the gradient of the function $.$

Note: The above algorithm is a descent algorithm to minimize and find a local minimum of a given function. This is the reason why we used the conversion $.$ If one were to re$-$write the algorithm without this conversion, we would have maximized and Step $1$ of the algorithm would be $.$ Such an algorithm is called a gradient ascent algorithm. It is more common in literature in optimization to use the descent version rather than the ascent version.

One Step of Gradient Ascent for the Poisson GLM

$2$ points possible $($graded$)$

Let and let $,$ for some constant $.$ For the Poisson GLM$,$ recall that and we have $.$

What is for any $?$

Use X$\_$i for and Y$\_$i for $.$

unanswered

$?$ STANDARD NOTATION

Let

$,,$

$,,$

$,$

Step size $.$

What is the new $?$