Convergence of the RMSProp deep learning method with penalty for nonconvex optimization.

A norm version of the RMSProp algorithm with penalty (termed RMSPropW) is introduced into the deep learning framework and its convergence is addressed both analytically and numerically. For rigour, we consider the general nonconvex setting and prove the boundedness and convergence of the RMSPropW method in both deterministic and stochastic cases. This equips us with strict upper bounds on both the moving average squared norm of the gradient and the norm of weight parameters throughout the learning process, owing to the penalty term within the proposed cost function. In the deterministic (batch) case, the boundedness of the moving average squared norm of the gradient is employed to prove that the gradient sequence converges to zero when using a fixed step size, while with diminishing stepsizes, the minimum of the gradient sequence converges to zero. In the stochastic case, due to the boundedness of the weight evolution sequence, it is further shown that the weight sequence converges to a stationary point with probability 1. Finally, illustrative simulations are provided to support the theoretical analysis, including a comparison with the standard RMSProp on MNIST, CIFAR-10, and IMDB datasets.