Reproducibility of neuroimaging analyses across operating systems
Neuroimaging pipelines are known to generate different results depending on the computing platform where they are compiled and executed. We quantify these differences for brain tissue classification, fMRI analysis, and cortical thickness (CT) extraction, using three of the main neuroimaging packages...
Main Authors: | , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2015-04-01
|
Series: | Frontiers in Neuroinformatics |
Subjects: | |
Online Access: | http://journal.frontiersin.org/Journal/10.3389/fninf.2015.00012/full |
id |
doaj-ebbd0e50e04f42d28f0fa4cc09f24af8 |
---|---|
record_format |
Article |
spelling |
doaj-ebbd0e50e04f42d28f0fa4cc09f24af82020-11-25T00:54:43ZengFrontiers Media S.A.Frontiers in Neuroinformatics1662-51962015-04-01910.3389/fninf.2015.00012135293Reproducibility of neuroimaging analyses across operating systemsTristan eGlatard0Tristan eGlatard1Lindsay Burke Lewis2Rafael Ferreira da Silva3Reza eAdalat4Natacha eBeck5Claude eLepage6Pierre eRioux7Marc-Etienne eRousseau8Tarek eSherif9Ewa eDeelman10Najmeh eKhalili-Mahani11Alan Charles Evans12McConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityUniversity of Lyon, CNRS, INSERM, CREATISMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityUniversity of Southern California, Information Sciences InstituteMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityUniversity of Southern California, Information Sciences InstituteMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityMcConnell Brain Imaging Centre, Montreal Neurological Institute, McGill UniversityNeuroimaging pipelines are known to generate different results depending on the computing platform where they are compiled and executed. We quantify these differences for brain tissue classification, fMRI analysis, and cortical thickness (CT) extraction, using three of the main neuroimaging packages (FSL, Freesurfer and CIVET) and different versions of GNU/Linux. We also identify some causes of these differences using library and system call interception. We find that these packages use mathematical functions based on single-precision floating-point arithmetic whose implementations in operating systems continue to evolve. While these differences have little or no impact on simple analysis pipelines such as brain extraction and cortical tissue classification, their accumulation creates important differences in longer pipelines such as subcortical tissue classification, fMRI analysis, and cortical thickness extraction. With FSL, most Dice coefficients between subcortical classifications obtained on different operating systems remain above 0.9, but values as low as 0.59 are observed. Independent component analyses (ICA) of fMRI data differ between operating systems in one third of the tested subjects, due to differences in motion correction. With Freesurfer and CIVET, in some brain regions we find an effect of build or operating system on cortical thickness. A first step to correct these reproducibility issues would be to use more precise representations of floating-point numbers in the critical sections of the pipelines. The numerical stability of pipelines should also be reviewed.http://journal.frontiersin.org/Journal/10.3389/fninf.2015.00012/fullFreeSurferreproducibilityOperating SystemsfslCivet |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Tristan eGlatard Tristan eGlatard Lindsay Burke Lewis Rafael Ferreira da Silva Reza eAdalat Natacha eBeck Claude eLepage Pierre eRioux Marc-Etienne eRousseau Tarek eSherif Ewa eDeelman Najmeh eKhalili-Mahani Alan Charles Evans |
spellingShingle |
Tristan eGlatard Tristan eGlatard Lindsay Burke Lewis Rafael Ferreira da Silva Reza eAdalat Natacha eBeck Claude eLepage Pierre eRioux Marc-Etienne eRousseau Tarek eSherif Ewa eDeelman Najmeh eKhalili-Mahani Alan Charles Evans Reproducibility of neuroimaging analyses across operating systems Frontiers in Neuroinformatics FreeSurfer reproducibility Operating Systems fsl Civet |
author_facet |
Tristan eGlatard Tristan eGlatard Lindsay Burke Lewis Rafael Ferreira da Silva Reza eAdalat Natacha eBeck Claude eLepage Pierre eRioux Marc-Etienne eRousseau Tarek eSherif Ewa eDeelman Najmeh eKhalili-Mahani Alan Charles Evans |
author_sort |
Tristan eGlatard |
title |
Reproducibility of neuroimaging analyses across operating systems |
title_short |
Reproducibility of neuroimaging analyses across operating systems |
title_full |
Reproducibility of neuroimaging analyses across operating systems |
title_fullStr |
Reproducibility of neuroimaging analyses across operating systems |
title_full_unstemmed |
Reproducibility of neuroimaging analyses across operating systems |
title_sort |
reproducibility of neuroimaging analyses across operating systems |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Neuroinformatics |
issn |
1662-5196 |
publishDate |
2015-04-01 |
description |
Neuroimaging pipelines are known to generate different results depending on the computing platform where they are compiled and executed. We quantify these differences for brain tissue classification, fMRI analysis, and cortical thickness (CT) extraction, using three of the main neuroimaging packages (FSL, Freesurfer and CIVET) and different versions of GNU/Linux. We also identify some causes of these differences using library and system call interception. We find that these packages use mathematical functions based on single-precision floating-point arithmetic whose implementations in operating systems continue to evolve. While these differences have little or no impact on simple analysis pipelines such as brain extraction and cortical tissue classification, their accumulation creates important differences in longer pipelines such as subcortical tissue classification, fMRI analysis, and cortical thickness extraction. With FSL, most Dice coefficients between subcortical classifications obtained on different operating systems remain above 0.9, but values as low as 0.59 are observed. Independent component analyses (ICA) of fMRI data differ between operating systems in one third of the tested subjects, due to differences in motion correction. With Freesurfer and CIVET, in some brain regions we find an effect of build or operating system on cortical thickness. A first step to correct these reproducibility issues would be to use more precise representations of floating-point numbers in the critical sections of the pipelines. The numerical stability of pipelines should also be reviewed. |
topic |
FreeSurfer reproducibility Operating Systems fsl Civet |
url |
http://journal.frontiersin.org/Journal/10.3389/fninf.2015.00012/full |
work_keys_str_mv |
AT tristaneglatard reproducibilityofneuroimaginganalysesacrossoperatingsystems AT tristaneglatard reproducibilityofneuroimaginganalysesacrossoperatingsystems AT lindsayburkelewis reproducibilityofneuroimaginganalysesacrossoperatingsystems AT rafaelferreiradasilva reproducibilityofneuroimaginganalysesacrossoperatingsystems AT rezaeadalat reproducibilityofneuroimaginganalysesacrossoperatingsystems AT natachaebeck reproducibilityofneuroimaginganalysesacrossoperatingsystems AT claudeelepage reproducibilityofneuroimaginganalysesacrossoperatingsystems AT pierreerioux reproducibilityofneuroimaginganalysesacrossoperatingsystems AT marcetienneerousseau reproducibilityofneuroimaginganalysesacrossoperatingsystems AT tarekesherif reproducibilityofneuroimaginganalysesacrossoperatingsystems AT ewaedeelman reproducibilityofneuroimaginganalysesacrossoperatingsystems AT najmehekhalilimahani reproducibilityofneuroimaginganalysesacrossoperatingsystems AT alancharlesevans reproducibilityofneuroimaginganalysesacrossoperatingsystems |
_version_ |
1725233071564783616 |