Consider this hypothetical optimizer: nn+1#(ninitializedat0)gn1i=1N(;xi,yi)+nn1gg Arrows signify variable assignment and the above steps are repeated in a loop. is a parameter vector and is initialized randomly. How does this optimizer differ from SGD with momentum? This optimizer gives less weight to "important" gradient dimensions, which could slow time until convergence. This optimizer's running average gives more weight to older gradients, which could slow or prevent convergence. The magnitude of the optimizer's updates is guaranteed to converge to zero more rapidly