The Asynchronous Training Algorithm Based on Sampling and Mean Fusion for Distributed RNN

Training of large scale deep neural networks with distributed implementations is an effective way to improve the efficiency. However, high network communication cost for synchronizing gradients and parameters is a major bottleneck in distributed training. In this work, we propose an asynchronous tra...

Full description

Bibliographic Details

Published in:	IEEE Access
Main Authors:	Dejiao Niu, Tianquan Liu, Tao Cai, Shijie Zhou
Format:	Article
Language:	English
Published:	IEEE 2020-01-01
Subjects:	Asynchronous training distributed recurrent neural network mean fusion
Online Access:	https://ieeexplore.ieee.org/document/8827466/