Deep Net Tree Structure for Balance of Capacity and Approximation Ability

• Main Authors: Charles K. Chui, Shao-Bo Lin, Ding-Xuan Zhou
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2019-09-01
• Series: Frontiers in Applied Mathematics and Statistics
• Subjects: deep nets; learning theory; deep learning; tree structure; empirical risk minimization
• Online Access: https://www.frontiersin.org/article/10.3389/fams.2019.00046/full

Deep learning has been successfully used in various applications including image classification, natural language processing and game theory. The heart of deep learning is to adopt deep neural networks (deep nets for short) with certain structures to build up the estimator. Depth and structure of deep nets are two crucial factors in promoting the development of deep learning. In this paper, we propose a novel tree structure to equip deep nets to compensate the capacity drawback of deep fully connected neural networks (DFCN) and enhance the approximation ability of deep convolutional neural networks (DCNN). Based on an empirical risk minimization algorithm, we derive fast learning rates for deep nets.