Probabilistic Siamese Networks for Learning Representations

• Main Author: Liu, Chen
• Other Authors: Frey, Brendan J.
• Language: en_ca
• Published: 2013
• Subjects: Machine Learning; Neural Networks; 0544
• Online Access: http://hdl.handle.net/1807/43097

We explore the training of deep neural networks to produce vector representations using weakly labelled information in the form of binary similarity labels for pairs of training images. Previous methods such as siamese networks, IMAX and others, have used fixed cost functions such as $L_1$, $L_2$-norms and mutual information to drive the representations of similar images together and different images apart. In this work, we formulate learning as maximizing the likelihood of binary similarity labels for pairs of input images, under a parameterized probabilistic similarity model. We describe and evaluate several forms of the similarity model that account for false positives and false negatives differently. We extract representations of MNIST, AT\&T ORL and COIL-100 images and use them to obtain classification results. We compare these results with state-of-the-art techniques such as deep neural networks and convolutional neural networks. We also study our method from a dimensionality reduction prospective.