Unsupervised Feature Learning With Winner-Takes-All Based STDP

• Main Authors: Paul Ferré, Franck Mamalet, Simon J. Thorpe
• Format: Article
• Language: English
• Published: Frontiers Media S.A. 2018-04-01
• Series: Frontiers in Computational Neuroscience
• Subjects: Spike-Timing-Dependent-Pasticity; neural network; unsupervised learning; winner-takes-all; vision
• Online Access: http://journal.frontiersin.org/article/10.3389/fncom.2018.00024/full

We present a novel strategy for unsupervised feature learning in image applications inspired by the Spike-Timing-Dependent-Plasticity (STDP) biological learning rule. We show equivalence between rank order coding Leaky-Integrate-and-Fire neurons and ReLU artificial neurons when applied to non-temporal data. We apply this to images using rank-order coding, which allows us to perform a full network simulation with a single feed-forward pass using GPU hardware. Next we introduce a binary STDP learning rule compatible with training on batches of images. Two mechanisms to stabilize the training are also presented : a Winner-Takes-All (WTA) framework which selects the most relevant patches to learn from along the spatial dimensions, and a simple feature-wise normalization as homeostatic process. This learning process allows us to train multi-layer architectures of convolutional sparse features. We apply our method to extract features from the MNIST, ETH80, CIFAR-10, and STL-10 datasets and show that these features are relevant for classification. We finally compare these results with several other state of the art unsupervised learning methods.