Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations

Universal Function Approximation by Deep Neural Nets with Bounded Width and ReLU Activations

This article concerns the expressive power of depth in neural nets with ReLU activations and a bounded width. We are particularly interested in the following questions: What is the minimal width <inline-formula> <math display="inline"> <semantics> <mrow> <msub>...

Full description

Bibliographic Details
Main Author:	Boris Hanin
Format:	Article
Language:	English
Published:	MDPI AG 2019-10-01
Series:	Mathematics
Subjects:	deep neural nets relu networks approximation theory
Online Access:	https://www.mdpi.com/2227-7390/7/10/992

Similar Items

ReLU Network with Bounded Width Is a Universal Approximator in View of an Approximate Identity
by: Sunghwan Moon
Published: (2021-01-01)

A new scheme for training ReLU-based multi-layer feedforward neural networks
by: Wang, Hao
Published: (2017)

Quantum ReLU activation for Convolutional Neural Networks to improve diagnosis of Parkinson’s disease and COVID-19
by: Parisi, Luca, et al.
Published: (2021)

PARAMETRIC FLATTEN-T SWISH: AN ADAPTIVE NONLINEAR ACTIVATION FUNCTION FOR DEEP LEARNING
by: Hock Hung Chieng, et al.
Published: (2020-11-01)

Flatten-T Swish: a thresholded ReLU-Swish-like activation function for deep learning
by: Hock Hung Chieng, et al.
Published: (2018-07-01)

Learning algorithm analysis for deep neural network with ReLu activation functions
by: Płaczek Stanisław, et al.
Published: (2018-01-01)

An Integrated Design for Classification and Localization of Diabetic Foot Ulcer Based on CNN and YOLOv2-DFU Models
by: Javaria Amin, et al.
Published: (2020-01-01)

Modeling of Convolutional Neural Network Architecture for Recognizing The Pandava Mask
by: Andi Sanjaya, et al.
Published: (2020-07-01)

ERDNN: Error-Resilient Deep Neural Networks With a New Error Correction Layer and Piece-Wise Rectified Linear Unit
by: Muhammad Salman Ali, et al.
Published: (2020-01-01)

Studying Perturbations on the Input of Two-Layer Neural Networks with ReLU Activation
by: Alsubaihi, Salman
Published: (2019)

A Global Universality of Two-Layer Neural Networks with ReLU Activations
by: Naoya Hatano, et al.
Published: (2021-01-01)

Convolutional Neural Network with a Learnable Spatial Activation Function for SAR Image Despeckling and Forest Image Analysis
by: Hao Wang, et al.
Published: (2021-08-01)

Gaussian Perturbations in ReLU Networks and the Arrangement of Activation Regions
by: Daróczy, B.
Published: (2022)

Towards Fast computation of certified robustness for ReLU networks
by: Weng, Tsui-Wei, et al.
Published: (2021)

Deep Net Tree Structure for Balance of Capacity and Approximation Ability
by: Charles K. Chui, et al.
Published: (2019-09-01)

On the Approximation of the Resource Equivalences in Petri Nets with the Invisible Transitions
by: Vladimir A. Bashkin
Published: (2020-06-01)

Understanding a Block of Layers in Deep Neural Networks: Optimization, Probabilistic and Tropical Geometric Perspectives
by: Bibi, Adel
Published: (2020)

Deep Neural Network Based Demand Side Short Term Load Forecasting
by: Seunghyoung Ryu, et al.
Published: (2016-12-01)

Mathematical Analysis of DCN-Based Super-Resolution
by: C. Lee, et al.
Published: (2020-01-01)

Impact of Approximate Multipliers on VGG Deep Learning Network
by: Issam Hammad, et al.
Published: (2018-01-01)

Deep Nets for Local Manifold Learning
by: Charles K. Chui, et al.
Published: (2018-05-01)

Implementation of Perceptron as an Essential Element of Configurable Neural Networks
by: Chao Geng, et al.
Published: (2020-07-01)

Small ReLU networks are powerful memorizers: A tight analysis of memorization capacity
by: Yun, Chulhee, et al.
Published: (2021)

RMAF: Relu-Memristor-Like Activation Function for Deep Learning
by: Yongbin Yu, et al.
Published: (2020-01-01)

StochasticNet: Forming Deep Neural Networks via Stochastic Connectivity
by: Mohammad Javad Shafiee, et al.
Published: (2016-01-01)

BERT-Based Chinese Relation Extraction for Public Security
by: Jiaqi Hou, et al.
Published: (2020-01-01)

Smooth Function Approximation by Deep Neural Networks with General Activation Functions
by: Ilsang Ohn, et al.
Published: (2019-06-01)

Handling Vanishing Gradient Problem Using Artificial Derivative
by: Zheng Hu, et al.
Published: (2021-01-01)

A New Deep Learning Model for The Classification of Poisonous and Edible Mushrooms Based on Improved AlexNet Convolutional Neural Network
by: Boonlue, S., et al.
Published: (2022)

The Cramming, Softening and Integrating Learning Algorithm with ReLU activation function for Binary Input/Output Problems
by: Tsai, Yu-Han, et al.
Published: (2019)

Block LU simulation with an Approximate Model of Coregionalization
by: Wang, Tong
Published: (2009)

Stream of Unbalanced Medical Big Data Using Convolutional Neural Network
by: Weiwei Gao, et al.
Published: (2020-01-01)

Hyperparameter optimisation for Capsule Networks
by: Gagana B, et al.
Published: (2019-03-01)

Deep web, dark web, dark net
by: Masayuki Hatta
Published: (2020-12-01)

Exploring Per-Input Filter Selection and Approximation Techniques for Deep Neural Networks
by: Gaur, Yamini
Published: (2019)

An Ultrashort-Term Net Load Forecasting Model Based on Phase Space Reconstruction and Deep Neural Network
by: Fei Mei, et al.
Published: (2019-04-01)

The Cramming, Softening and Integrating Learning Algorithm with ReLU Activation Function for Real Number Input / Output Problems
by: Liang, Wei-Ting, et al.
Published: (2019)

ECRU: An Encoder-Decoder Based Convolution Neural Network (CNN) for Road-Scene Understanding
by: Robail Yasrab
Published: (2018-10-01)

Blade Rub-Impact Fault Identification Using Autoencoder-Based Nonlinear Function Approximation and a Deep Neural Network
by: Alexander E. Prosvirin, et al.
Published: (2020-11-01)

LU-NET: An Improved U-Net for Ventricular Segmentation
by: Jun Zhang, et al.
Published: (2019-01-01)