Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up
Background: Some patients with acromegaly do not reach the remission standard in the short term after surgery but achieve remission without additional postoperative treatment during long-term follow-up; this phenomenon is defined as postoperative delayed remission (DR). DR may complicate the interpr...
Main Authors: | , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2020-09-01
|
Series: | Frontiers in Endocrinology |
Subjects: | |
Online Access: | https://www.frontiersin.org/article/10.3389/fendo.2020.00643/full |
id |
doaj-1b9b6f6af45548fdbc74bf58037c4b5f |
---|---|
record_format |
Article |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Congxin Dai Yanghua Fan Yichao Li Xinjie Bao Yansheng Li Mingliang Su Yong Yao Kan Deng Bing Xing Feng Feng Ming Feng Renzhi Wang |
spellingShingle |
Congxin Dai Yanghua Fan Yichao Li Xinjie Bao Yansheng Li Mingliang Su Yong Yao Kan Deng Bing Xing Feng Feng Ming Feng Renzhi Wang Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up Frontiers in Endocrinology acromegaly delayed remission machine learning LIME SHAP |
author_facet |
Congxin Dai Yanghua Fan Yichao Li Xinjie Bao Yansheng Li Mingliang Su Yong Yao Kan Deng Bing Xing Feng Feng Ming Feng Renzhi Wang |
author_sort |
Congxin Dai |
title |
Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up |
title_short |
Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up |
title_full |
Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up |
title_fullStr |
Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up |
title_full_unstemmed |
Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-Up |
title_sort |
development and interpretation of multiple machine learning models for predicting postoperative delayed remission of acromegaly patients during long-term follow-up |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Endocrinology |
issn |
1664-2392 |
publishDate |
2020-09-01 |
description |
Background: Some patients with acromegaly do not reach the remission standard in the short term after surgery but achieve remission without additional postoperative treatment during long-term follow-up; this phenomenon is defined as postoperative delayed remission (DR). DR may complicate the interpretation of surgical outcomes in patients with acromegaly and interfere with decision-making regarding postoperative adjuvant therapy.Objective: We aimed to develop and validate machine learning (ML) models for predicting DR in acromegaly patients who have not achieved remission within 6 months of surgery.Methods: We enrolled 306 acromegaly patients and randomly divided them into training and test datasets. We used the recursive feature elimination (RFE) algorithm to select features and applied six ML algorithms to construct DR prediction models. The performance of these ML models was validated using receiver operating characteristics analysis. We used permutation importance, SHapley Additive exPlanations (SHAP), and local interpretable model–agnostic explanation (LIME) algorithms to determine the importance of the selected features and interpret the ML models.Results: Fifty-five (17.97%) acromegaly patients met the criteria for DR, and five features (post-1w rGH, post-1w nGH, post-6m rGH, post-6m IGF-1, and post-6m nGH) were significantly associated with DR in both the training and the test datasets. After the RFE feature selection, the XGboost model, which comprised the 15 important features, had the greatest discriminatory ability (area under the curve = 0.8349, sensitivity = 0.8889, Youden's index = 0.6842). The XGboost model showed good discrimination ability and provided significantly better estimates of DR of patients with acromegaly compared with using only the Knosp grade. The results obtained from permutation importance, SHAP, and LIME algorithms showed that post-6m IGF-1 is the most important feature in XGboost algorithm prediction and showed the reliability and the clinical practicability of the XGboost model in DR prediction.Conclusions: ML-based models can serve as an effective non-invasive approach to predicting DR and could aid in determining individual treatment and follow-up strategies for acromegaly patients who have not achieved remission within 6 months of surgery. |
topic |
acromegaly delayed remission machine learning LIME SHAP |
url |
https://www.frontiersin.org/article/10.3389/fendo.2020.00643/full |
work_keys_str_mv |
AT congxindai developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT yanghuafan developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT yichaoli developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT xinjiebao developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT yanshengli developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT mingliangsu developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT yongyao developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT kandeng developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT bingxing developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT fengfeng developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT mingfeng developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup AT renzhiwang developmentandinterpretationofmultiplemachinelearningmodelsforpredictingpostoperativedelayedremissionofacromegalypatientsduringlongtermfollowup |
_version_ |
1724470945496694784 |
spelling |
doaj-1b9b6f6af45548fdbc74bf58037c4b5f2020-11-25T03:55:03ZengFrontiers Media S.A.Frontiers in Endocrinology1664-23922020-09-011110.3389/fendo.2020.00643547974Development and Interpretation of Multiple Machine Learning Models for Predicting Postoperative Delayed Remission of Acromegaly Patients During Long-Term Follow-UpCongxin Dai0Yanghua Fan1Yichao Li2Xinjie Bao3Yansheng Li4Mingliang Su5Yong Yao6Kan Deng7Bing Xing8Feng Feng9Ming Feng10Renzhi Wang11Department of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDHC Mediway Technology Co., Ltd., Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDHC Mediway Technology Co., Ltd., Beijing, ChinaDHC Mediway Technology Co., Ltd., Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Radiology, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaDepartment of Neurosurgery, Peking Union Medical College Hospital, Chinese Academy of Medical Sciences and Peking Union Medical College, Beijing, ChinaBackground: Some patients with acromegaly do not reach the remission standard in the short term after surgery but achieve remission without additional postoperative treatment during long-term follow-up; this phenomenon is defined as postoperative delayed remission (DR). DR may complicate the interpretation of surgical outcomes in patients with acromegaly and interfere with decision-making regarding postoperative adjuvant therapy.Objective: We aimed to develop and validate machine learning (ML) models for predicting DR in acromegaly patients who have not achieved remission within 6 months of surgery.Methods: We enrolled 306 acromegaly patients and randomly divided them into training and test datasets. We used the recursive feature elimination (RFE) algorithm to select features and applied six ML algorithms to construct DR prediction models. The performance of these ML models was validated using receiver operating characteristics analysis. We used permutation importance, SHapley Additive exPlanations (SHAP), and local interpretable model–agnostic explanation (LIME) algorithms to determine the importance of the selected features and interpret the ML models.Results: Fifty-five (17.97%) acromegaly patients met the criteria for DR, and five features (post-1w rGH, post-1w nGH, post-6m rGH, post-6m IGF-1, and post-6m nGH) were significantly associated with DR in both the training and the test datasets. After the RFE feature selection, the XGboost model, which comprised the 15 important features, had the greatest discriminatory ability (area under the curve = 0.8349, sensitivity = 0.8889, Youden's index = 0.6842). The XGboost model showed good discrimination ability and provided significantly better estimates of DR of patients with acromegaly compared with using only the Knosp grade. The results obtained from permutation importance, SHAP, and LIME algorithms showed that post-6m IGF-1 is the most important feature in XGboost algorithm prediction and showed the reliability and the clinical practicability of the XGboost model in DR prediction.Conclusions: ML-based models can serve as an effective non-invasive approach to predicting DR and could aid in determining individual treatment and follow-up strategies for acromegaly patients who have not achieved remission within 6 months of surgery.https://www.frontiersin.org/article/10.3389/fendo.2020.00643/fullacromegalydelayed remissionmachine learningLIMESHAP |