A Zeroth-Order Adaptive Learning Rate Method to Reduce Cost of Hyperparameter Tuning for Deep Learning

Due to powerful data representation ability, deep learning has dramatically improved the state-of-the-art in many practical applications. However, the utility highly depends on fine-tuning of hyper-parameters, including learning rate, batch size, and network initialization. Although many first-order...

Full description

Bibliographic Details

Published in:	Applied Sciences
Main Authors:	Yanan Li, Xuebin Ren, Fangyuan Zhao, Shusen Yang
Format:	Article
Language:	English
Published:	MDPI AG 2021-10-01
Subjects:	deep learning adaptive learning rate robustness stochastic gradient descent
Online Access:	https://www.mdpi.com/2076-3417/11/21/10184