Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning
Minimally invasive surgery (MIS) is among the preferred procedures for treating a number of ailments as patients benefit from fast recovery and reduced blood loss. The trade-off is that surgeons lose direct visual contact with the surgical site and have limited intra-operative imaging techniques for...
Main Authors: | , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
IEEE
2020-01-01
|
Series: | IEEE Access |
Subjects: | |
Online Access: | https://ieeexplore.ieee.org/document/9032130/ |
id |
doaj-db2933994d55475c920a1313d13b3c43 |
---|---|
record_format |
Article |
spelling |
doaj-db2933994d55475c920a1313d13b3c432021-03-30T02:12:24ZengIEEEIEEE Access2169-35362020-01-018518535186110.1109/ACCESS.2020.29800259032130Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep LearningYaqub Jonmohamadi0https://orcid.org/0000-0002-2912-0554Yu Takeda1Fengbei Liu2Fumio Sasazawa3Gabriel Maicas4Ross Crawford5Jonathan Roberts6Ajay K. Pandey7Gustavo Carneiro8School of Electrical Engineering and Robotics, Science and Engineering Faculty, Queensland University of Technology, Brisbane, QLD, AustraliaDepartment of Orthopaedic Surgery, Hyogo College of Medicine, Nishinomiya, JapanSchool of Computer Science, Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, AustraliaFaculty of Medicine and Graduate School of Medicine, Hokkaido University, Sapporo, JapanSchool of Computer Science, Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, AustraliaInstitute of Health and Biomedical Innovation, Queensland University of Technology, Brisbane, QLD, AustraliaSchool of Electrical Engineering and Robotics, Science and Engineering Faculty, Queensland University of Technology, Brisbane, QLD, AustraliaSchool of Electrical Engineering and Robotics, Science and Engineering Faculty, Queensland University of Technology, Brisbane, QLD, AustraliaSchool of Computer Science, Australian Institute for Machine Learning, The University of Adelaide, Adelaide, SA, AustraliaMinimally invasive surgery (MIS) is among the preferred procedures for treating a number of ailments as patients benefit from fast recovery and reduced blood loss. The trade-off is that surgeons lose direct visual contact with the surgical site and have limited intra-operative imaging techniques for real-time feedback. Computer vision methods as well as segmentation and tracking of the tissues and tools in the video frames, are increasingly being adopted to MIS to alleviate such limitations. So far, most of the advances in MIS have been focused on laparoscopic applications, with scarce literature on knee arthroscopy. Here for the first time, we propose a new method for the automatic segmentation of multiple tissue structures for knee arthroscopy. The training data of 3868 images were collected from 4 cadaver experiments, 5 knees, and manually contoured by two clinicians into four classes: Femur, Anterior Cruciate Ligament (ACL), Tibia, and Meniscus. Our approach adapts the U-net and the U-net++ architectures for this segmentation task. Using the cross-validation experiment, the mean Dice similarity coefficients for Femur, Tibia, ACL, and Meniscus are 0.78, 0.50, 0.41, 0.43 using the U-net and 0.79, 0.50, 0.51, 0.48 using the U-net++. While the reported segmentation method is of great applicability in terms of contextual awareness for the surgical team, it can also be used for medical robotic applications such as SLAM and depth mapping.https://ieeexplore.ieee.org/document/9032130/Arthroscopyartificial intelligenceauto segmentationdeep learningendoscopysurgery |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Yaqub Jonmohamadi Yu Takeda Fengbei Liu Fumio Sasazawa Gabriel Maicas Ross Crawford Jonathan Roberts Ajay K. Pandey Gustavo Carneiro |
spellingShingle |
Yaqub Jonmohamadi Yu Takeda Fengbei Liu Fumio Sasazawa Gabriel Maicas Ross Crawford Jonathan Roberts Ajay K. Pandey Gustavo Carneiro Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning IEEE Access Arthroscopy artificial intelligence auto segmentation deep learning endoscopy surgery |
author_facet |
Yaqub Jonmohamadi Yu Takeda Fengbei Liu Fumio Sasazawa Gabriel Maicas Ross Crawford Jonathan Roberts Ajay K. Pandey Gustavo Carneiro |
author_sort |
Yaqub Jonmohamadi |
title |
Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning |
title_short |
Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning |
title_full |
Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning |
title_fullStr |
Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning |
title_full_unstemmed |
Automatic Segmentation of Multiple Structures in Knee Arthroscopy Using Deep Learning |
title_sort |
automatic segmentation of multiple structures in knee arthroscopy using deep learning |
publisher |
IEEE |
series |
IEEE Access |
issn |
2169-3536 |
publishDate |
2020-01-01 |
description |
Minimally invasive surgery (MIS) is among the preferred procedures for treating a number of ailments as patients benefit from fast recovery and reduced blood loss. The trade-off is that surgeons lose direct visual contact with the surgical site and have limited intra-operative imaging techniques for real-time feedback. Computer vision methods as well as segmentation and tracking of the tissues and tools in the video frames, are increasingly being adopted to MIS to alleviate such limitations. So far, most of the advances in MIS have been focused on laparoscopic applications, with scarce literature on knee arthroscopy. Here for the first time, we propose a new method for the automatic segmentation of multiple tissue structures for knee arthroscopy. The training data of 3868 images were collected from 4 cadaver experiments, 5 knees, and manually contoured by two clinicians into four classes: Femur, Anterior Cruciate Ligament (ACL), Tibia, and Meniscus. Our approach adapts the U-net and the U-net++ architectures for this segmentation task. Using the cross-validation experiment, the mean Dice similarity coefficients for Femur, Tibia, ACL, and Meniscus are 0.78, 0.50, 0.41, 0.43 using the U-net and 0.79, 0.50, 0.51, 0.48 using the U-net++. While the reported segmentation method is of great applicability in terms of contextual awareness for the surgical team, it can also be used for medical robotic applications such as SLAM and depth mapping. |
topic |
Arthroscopy artificial intelligence auto segmentation deep learning endoscopy surgery |
url |
https://ieeexplore.ieee.org/document/9032130/ |
work_keys_str_mv |
AT yaqubjonmohamadi automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT yutakeda automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT fengbeiliu automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT fumiosasazawa automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT gabrielmaicas automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT rosscrawford automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT jonathanroberts automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT ajaykpandey automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning AT gustavocarneiro automaticsegmentationofmultiplestructuresinkneearthroscopyusingdeeplearning |
_version_ |
1724185680369680384 |