Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation
Background and objective: The analysis of retinal vessels in fundus images is vital in the diagnosis of retinal diseases and early diagnosis of chronic vascular diseases and diabetes. Automatic vessel segmentation relies on costly expert annotations that may have limitations in the level of detail....
Main Authors: | , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9050794/ |
id |
doaj-6434a3f5d71144129df9a85874a01193 |
---|---|
record_format |
Article |
spelling |
doaj-6434a3f5d71144129df9a85874a011932021-03-30T01:34:50ZengIEEEIEEE Access2169-35362020-01-018637576376910.1109/ACCESS.2020.29843729050794Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel SegmentationKyoung Jin Noh0https://orcid.org/0000-0002-0003-0387Jooyoung Kim1https://orcid.org/0000-0001-7771-5133Sang Jun Park2https://orcid.org/0000-0003-0542-2758Soochahn Lee3https://orcid.org/0000-0002-2975-2519Department of Ophthalmology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam, South KoreaDepartment of Ophthalmology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam, South KoreaDepartment of Ophthalmology, Seoul National University College of Medicine, Seoul National University Bundang Hospital, Seongnam, South KoreaSchool of Electrical Engineering, Kookmin University, Seoul, South KoreaBackground and objective: The analysis of retinal vessels in fundus images is vital in the diagnosis of retinal diseases and early diagnosis of chronic vascular diseases and diabetes. Automatic vessel segmentation relies on costly expert annotations that may have limitations in the level of detail. We develop an automatic method to generate highly-accurate vessel segmentation including fine-scale vessels. Methods: We present a new framework for fine-scale vessel segmentation from fundus images through registration and segmentation of corresponding fluorescein angiography (FA) images. We first register and aggregate the extracted vessels highlighted from the fluorescent dye in the FA frames. This FA vessel mask is then registered to the fundus image based on an initial fundus vessel mask. Post-processing is performed to refine the final vessel mask. Registration of the FA frames, and registration of FA vessel mask to the fundus image, are performed by similar coarse-to-fine hierarchical frameworks comprising both projective and deformable registration. Two convolutional neural networks with identical network structures, both trained on public datasets but with different configurations, are used for vessel segmentation of both the FA frames and the fundus images. Results: Qualitative examples support the robustness and accuracy of the proposed method. Quantitative evaluations, including the area-under-curve (AUC) of the receiver operating characteristic (ROC) curve are presented. Although fair comparisons cannot be made due to a lack of similar methods and adequate public datasets, we demonstrate that the proposed method with an AUC ROC of 0.979, outperforms a state-of-the-art automatic vessel segmentation method trained on publicly available datasets at 0.956. Conclusions: The proposed method generates accurate vessel segmentation results containing filamentary vessels that are virtually indiscernible to the naked eye in color retinal fundus images.https://ieeexplore.ieee.org/document/9050794/Fundus imagesfluorescein angiographymulti-modal registrationvessel segmentation |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Kyoung Jin Noh Jooyoung Kim Sang Jun Park Soochahn Lee |
spellingShingle |
Kyoung Jin Noh Jooyoung Kim Sang Jun Park Soochahn Lee Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation IEEE Access Fundus images fluorescein angiography multi-modal registration vessel segmentation |
author_facet |
Kyoung Jin Noh Jooyoung Kim Sang Jun Park Soochahn Lee |
author_sort |
Kyoung Jin Noh |
title |
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation |
title_short |
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation |
title_full |
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation |
title_fullStr |
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation |
title_full_unstemmed |
Multimodal Registration of Fundus Images With Fluorescein Angiography for Fine-Scale Vessel Segmentation |
title_sort |
multimodal registration of fundus images with fluorescein angiography for fine-scale vessel segmentation |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
Background and objective: The analysis of retinal vessels in fundus images is vital in the diagnosis of retinal diseases and early diagnosis of chronic vascular diseases and diabetes. Automatic vessel segmentation relies on costly expert annotations that may have limitations in the level of detail. We develop an automatic method to generate highly-accurate vessel segmentation including fine-scale vessels. Methods: We present a new framework for fine-scale vessel segmentation from fundus images through registration and segmentation of corresponding fluorescein angiography (FA) images. We first register and aggregate the extracted vessels highlighted from the fluorescent dye in the FA frames. This FA vessel mask is then registered to the fundus image based on an initial fundus vessel mask. Post-processing is performed to refine the final vessel mask. Registration of the FA frames, and registration of FA vessel mask to the fundus image, are performed by similar coarse-to-fine hierarchical frameworks comprising both projective and deformable registration. Two convolutional neural networks with identical network structures, both trained on public datasets but with different configurations, are used for vessel segmentation of both the FA frames and the fundus images. Results: Qualitative examples support the robustness and accuracy of the proposed method. Quantitative evaluations, including the area-under-curve (AUC) of the receiver operating characteristic (ROC) curve are presented. Although fair comparisons cannot be made due to a lack of similar methods and adequate public datasets, we demonstrate that the proposed method with an AUC ROC of 0.979, outperforms a state-of-the-art automatic vessel segmentation method trained on publicly available datasets at 0.956. Conclusions: The proposed method generates accurate vessel segmentation results containing filamentary vessels that are virtually indiscernible to the naked eye in color retinal fundus images. |
topic |
Fundus images fluorescein angiography multi-modal registration vessel segmentation |
url |
https://ieeexplore.ieee.org/document/9050794/ |
work_keys_str_mv |
AT kyoungjinnoh multimodalregistrationoffundusimageswithfluoresceinangiographyforfinescalevesselsegmentation AT jooyoungkim multimodalregistrationoffundusimageswithfluoresceinangiographyforfinescalevesselsegmentation AT sangjunpark multimodalregistrationoffundusimageswithfluoresceinangiographyforfinescalevesselsegmentation AT soochahnlee multimodalregistrationoffundusimageswithfluoresceinangiographyforfinescalevesselsegmentation |
_version_ |
1724186789767282688 |