Stability of steepest descent with momentum for quadratic functions.

This paper analyzes the effect of momentum on steepest descent training for quadratic performance functions. We demonstrate that there always exists a momentum coefficient that will stabilize the steepest descent algorithm, regardless of the value of the learning rate. We also demonstrate how the value of the momentum coefficient changes the convergence properties of the algorithm.