Influence Area of Overlap Singularity in Multilayer Perceptrons

• Main Authors: Weili Guo, Yuan Yang, Yingjiang Zhou, Yushun Tan, Haikun Wei, Aiguo Song, Guochen Pang
• Format: Article
• Language: English
• Published: IEEE 2018-01-01
• Series: IEEE Access
• Subjects: Multilayer perceptrons; dynamics; information geometry; overlap singularity; influence area
• Online Access: https://ieeexplore.ieee.org/document/8481340/

The existing overlap singularities in the parameter space significantly affect the learning dynamics of the multilayer perceptrons. From the obtained theoretical learning trajectories near overlap singularity, when the learning process has been affected by the overlap singularity, the influence area of the overlap singularity is just the line space where the two hidden units equal to each other. However, in the practical applications, different case has been observed and the influence area of such singularity may be larger. By analyzing the generalization error of multilayer perceptrons, we find that the error surface is much flatter near overlap singularity and the singularity would have much larger influence area. Finally, the validity of the obtained results are verified by taking an artificial experiment and two real-data experiments.