Fast deep neural correspondence for tracking and identifying neurons in C. elegans using semi-synthetic training

• Main Authors: Xinwei Yu, Matthew S Creamer, Francesco Randi, Anuj K Sharma, Scott W Linderman, Andrew M Leifer
• Format: Article
• Language: English
• Published: eLife Sciences Publications Ltd 2021-07-01
• Series: eLife
• Subjects: computer vision; deep learning; artificial neural network; tracking; registration; transformer
• Online Access: https://elifesciences.org/articles/66410

We present an automated method to track and identify neurons in C. elegans, called ‘fast Deep Neural Correspondence’ or fDNC, based on the transformer network architecture. The model is trained once on empirically derived semi-synthetic data and then predicts neural correspondence across held-out real animals. The same pre-trained model both tracks neurons across time and identifies corresponding neurons across individuals. Performance is evaluated against hand-annotated datasets, including NeuroPAL (Yemini et al., 2021). Using only position information, the method achieves 79.1% accuracy at tracking neurons within an individual and 64.1% accuracy at identifying neurons across individuals. Accuracy at identifying neurons across individuals is even higher (78.2%) when the model is applied to a dataset published by another group (Chaudhary et al., 2021). Accuracy reaches 74.7% on our dataset when using color information from NeuroPAL. Unlike previous methods, fDNC does not require straightening or transforming the animal into a canonical coordinate system. The method is fast and predicts correspondence in 10 ms making it suitable for future real-time applications.