Indeed, there's even a sense in which gradient descent is the optimal strategy for searching for a minimum. Let's suppose that we're trying to make a move v in position so as to decrease C as much as possible. This is equivalent to minimizing CCv. We'll constrain the size of the move so that v= for some small fixed >0. In other words, we want a move that is a small step of a fixed size, and we're trying to find the movement direction which decreases C as much as possible. It can be proved that the choice of v which minimizes Cv is v=C, where =/C is determined by the size constraint v=. So gradient descent can be viewed as a way of taking small steps in the direction which does the most to immediately decrease C. Exercises - Prove the assertion of the last paragraph. Hint: If you're not already familiar with the Cauchy-Schwarz inequality, you may find it helpful to familiarize yourself with it