Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging
Abstract One of the core challenges in applying machine learning and artificial intelligence to medicine is the limited availability of annotated medical data. Unlike in other applications of machine learning, where an abundance of labeled data is available, the labeling and annotation of medical da...
Main Authors: | , , , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Nature Publishing Group
2021-03-01
|
Series: | npj Digital Medicine |
Online Access: | https://doi.org/10.1038/s41746-021-00411-w |
id |
doaj-db547242a9544aa0933b586e70820352 |
---|---|
record_format |
Article |
spelling |
doaj-db547242a9544aa0933b586e708203522021-03-11T12:40:00ZengNature Publishing Groupnpj Digital Medicine2398-63522021-03-014111310.1038/s41746-021-00411-wAutomated identification of clinical features from sparsely annotated 3-dimensional medical imagingNadav Rakocz0Jeffrey N. Chiang1Muneeswar G. Nittala2Giulia Corradetti3Liran Tiosano4Swetha Velaga5Michael Thompson6Brian L. Hill7Sriram Sankararaman8Jonathan L. Haines9Margaret A. Pericak-Vance10Dwight Stambolian11Srinivas R. Sadda12Eran Halperin13Department of Computer Science, University of CaliforniaDepartment of Computational Medicine, University of CaliforniaDoheny Eye InstituteDoheny Eye InstituteDoheny Eye InstituteDoheny Eye InstituteDepartment of Computer Science, University of CaliforniaDepartment of Computer Science, University of CaliforniaDepartment of Computer Science, University of CaliforniaDepartment of Population & Quantitative Health Sciences, Case Western Reserve UniversityJohn P. Hussman Institute for Human Genomics, University of Miami Miller School of MedicineDepartment of Ophthalmology, University of Pennsylvania, Perelman School of MedicineDoheny Eye InstituteDepartment of Computer Science, University of CaliforniaAbstract One of the core challenges in applying machine learning and artificial intelligence to medicine is the limited availability of annotated medical data. Unlike in other applications of machine learning, where an abundance of labeled data is available, the labeling and annotation of medical data and images require a major effort of manual work by expert clinicians who do not have the time to annotate manually. In this work, we propose a new deep learning technique (SLIVER-net), to predict clinical features from 3-dimensional volumes using a limited number of manually annotated examples. SLIVER-net is based on transfer learning, where we borrow information about the structure and parameters of the network from publicly available large datasets. Since public volume data are scarce, we use 2D images and account for the 3-dimensional structure using a novel deep learning method which tiles the volume scans, and then adds layers that leverage the 3D structure. In order to illustrate its utility, we apply SLIVER-net to predict risk factors for progression of age-related macular degeneration (AMD), a leading cause of blindness, from optical coherence tomography (OCT) volumes acquired from multiple sites. SLIVER-net successfully predicts these factors despite being trained with a relatively small number of annotated volumes (hundreds) and only dozens of positive training examples. Our empirical evaluation demonstrates that SLIVER-net significantly outperforms standard state-of-the-art deep learning techniques used for medical volumes, and its performance is generalizable as it was validated on an external testing set. In a direct comparison with a clinician panel, we find that SLIVER-net also outperforms junior specialists, and identifies AMD progression risk factors similarly to expert retina specialists.https://doi.org/10.1038/s41746-021-00411-w |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Nadav Rakocz Jeffrey N. Chiang Muneeswar G. Nittala Giulia Corradetti Liran Tiosano Swetha Velaga Michael Thompson Brian L. Hill Sriram Sankararaman Jonathan L. Haines Margaret A. Pericak-Vance Dwight Stambolian Srinivas R. Sadda Eran Halperin |
spellingShingle |
Nadav Rakocz Jeffrey N. Chiang Muneeswar G. Nittala Giulia Corradetti Liran Tiosano Swetha Velaga Michael Thompson Brian L. Hill Sriram Sankararaman Jonathan L. Haines Margaret A. Pericak-Vance Dwight Stambolian Srinivas R. Sadda Eran Halperin Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging npj Digital Medicine |
author_facet |
Nadav Rakocz Jeffrey N. Chiang Muneeswar G. Nittala Giulia Corradetti Liran Tiosano Swetha Velaga Michael Thompson Brian L. Hill Sriram Sankararaman Jonathan L. Haines Margaret A. Pericak-Vance Dwight Stambolian Srinivas R. Sadda Eran Halperin |
author_sort |
Nadav Rakocz |
title |
Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
title_short |
Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
title_full |
Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
title_fullStr |
Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
title_full_unstemmed |
Automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
title_sort |
automated identification of clinical features from sparsely annotated 3-dimensional medical imaging |
publisher |
Nature Publishing Group |
series |
npj Digital Medicine |
issn |
2398-6352 |
publishDate |
2021-03-01 |
description |
Abstract One of the core challenges in applying machine learning and artificial intelligence to medicine is the limited availability of annotated medical data. Unlike in other applications of machine learning, where an abundance of labeled data is available, the labeling and annotation of medical data and images require a major effort of manual work by expert clinicians who do not have the time to annotate manually. In this work, we propose a new deep learning technique (SLIVER-net), to predict clinical features from 3-dimensional volumes using a limited number of manually annotated examples. SLIVER-net is based on transfer learning, where we borrow information about the structure and parameters of the network from publicly available large datasets. Since public volume data are scarce, we use 2D images and account for the 3-dimensional structure using a novel deep learning method which tiles the volume scans, and then adds layers that leverage the 3D structure. In order to illustrate its utility, we apply SLIVER-net to predict risk factors for progression of age-related macular degeneration (AMD), a leading cause of blindness, from optical coherence tomography (OCT) volumes acquired from multiple sites. SLIVER-net successfully predicts these factors despite being trained with a relatively small number of annotated volumes (hundreds) and only dozens of positive training examples. Our empirical evaluation demonstrates that SLIVER-net significantly outperforms standard state-of-the-art deep learning techniques used for medical volumes, and its performance is generalizable as it was validated on an external testing set. In a direct comparison with a clinician panel, we find that SLIVER-net also outperforms junior specialists, and identifies AMD progression risk factors similarly to expert retina specialists. |
url |
https://doi.org/10.1038/s41746-021-00411-w |
work_keys_str_mv |
AT nadavrakocz automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT jeffreynchiang automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT muneeswargnittala automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT giuliacorradetti automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT lirantiosano automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT swethavelaga automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT michaelthompson automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT brianlhill automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT sriramsankararaman automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT jonathanlhaines automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT margaretapericakvance automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT dwightstambolian automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT srinivasrsadda automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging AT eranhalperin automatedidentificationofclinicalfeaturesfromsparselyannotated3dimensionalmedicalimaging |
_version_ |
1724224199002685440 |