Provable convergence of Nesterov's accelerated gradient method for over-parameterized neural networks

Provable convergence of Nesterov's accelerated gradient method for over-parameterized neural networks

Momentum methods, such as heavy ball method (HB) and Nesterov's accelerated gradient method (NAG), have been widely used in training neural networks by incorporating the history of gradients into the current updating process. In practice, they often provide improved performance over (stochastic...

Full description

Bibliographic Details
Main Authors: Liu, X. (Author), Pan, Z. (Author), Tao, W. (Author)
Format: Article
Language:English
Published: Elsevier B.V. 2022
Subjects:
Convergence rates
Global conver-gence
Gradient methods
Gradient-descent
Gradient's methods
Multilayer neural networks
Nesterov accelerated gradient method
Nesterov's accelerated gradient method
Network layers
Neural networks
Neural tangent kernel
Neural-networks
Non-asymptotic
Non-asymptotic global convergence
Number theory
Overparameterization
Over-parameterization
Parameterization
Stochastic systems
Online Access:View Fulltext in Publisher

Internet

View Fulltext in Publisher

Similar Items