Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach
Abstract Background Histopathological classification of Wilms tumors determines treatment regimen. Machine learning has been shown to contribute to histopathological classification in various malignancies but requires large numbers of manually annotated images and thus specific pathological knowledg...
Main Authors: | , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
BMC
2021-08-01
|
Series: | Diagnostic Pathology |
Subjects: | |
Online Access: | https://doi.org/10.1186/s13000-021-01136-w |
id |
doaj-94fc23074ff94fc9bff4bcccf3f4ccb3 |
---|---|
record_format |
Article |
spelling |
doaj-94fc23074ff94fc9bff4bcccf3f4ccb32021-08-22T11:42:46ZengBMCDiagnostic Pathology1746-15962021-08-011611610.1186/s13000-021-01136-wInterobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approachJikke J. Rutgers0Tessa Bánki1Ananda van der Kamp2Tomas J. Waterlander3Marijn A. Scheijde-Vermeulen4Marry M. van den Heuvel-Eibrink5Jeroen A. W. M. van der Laak6Marta Fiocco7Annelies M. C. Mavinkurve-Groothuis8Ronald R. de Krijger9Princess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyDepartment of Pathology, Radboud University Medical CenterPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyPrincess Máxima Center for Pediatric OncologyAbstract Background Histopathological classification of Wilms tumors determines treatment regimen. Machine learning has been shown to contribute to histopathological classification in various malignancies but requires large numbers of manually annotated images and thus specific pathological knowledge. This study aimed to assess whether trained, inexperienced observers could contribute to reliable annotation of Wilms tumor components for classification performed by machine learning. Methods Four inexperienced observers (medical students) were trained in histopathology of normal kidneys and Wilms tumors by an experienced observer (pediatric pathologist). Twenty randomly selected scanned Wilms tumor-slides (from n = 1472 slides) were annotated, and annotations were independently classified by both the inexperienced observers and two experienced pediatric pathologists. Agreement between the six observers and for each tissue element was measured using kappa statistics (κ). Results Pairwise interobserver agreement between all inexperienced and experienced observers was high (range: 0.845–0.950). The interobserver variability for the different histological elements, including all vital tumor components and therapy-related effects, showed high values for all κ-coefficients (> 0.827). Conclusions Inexperienced observers can be trained to recognize specific histopathological tumor and tissue elements with high interobserver agreement with experienced observers. Nevertheless, supervision by experienced pathologists remains necessary. Results of this study can be used to facilitate more rapid progress for supervised machine learning-based algorithm development in pediatric pathology and beyond.https://doi.org/10.1186/s13000-021-01136-wWilms tumorInterobserver variabilityMachine learningHistopathologyClassificationAI (artificial intelligence) |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Jikke J. Rutgers Tessa Bánki Ananda van der Kamp Tomas J. Waterlander Marijn A. Scheijde-Vermeulen Marry M. van den Heuvel-Eibrink Jeroen A. W. M. van der Laak Marta Fiocco Annelies M. C. Mavinkurve-Groothuis Ronald R. de Krijger |
spellingShingle |
Jikke J. Rutgers Tessa Bánki Ananda van der Kamp Tomas J. Waterlander Marijn A. Scheijde-Vermeulen Marry M. van den Heuvel-Eibrink Jeroen A. W. M. van der Laak Marta Fiocco Annelies M. C. Mavinkurve-Groothuis Ronald R. de Krijger Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach Diagnostic Pathology Wilms tumor Interobserver variability Machine learning Histopathology Classification AI (artificial intelligence) |
author_facet |
Jikke J. Rutgers Tessa Bánki Ananda van der Kamp Tomas J. Waterlander Marijn A. Scheijde-Vermeulen Marry M. van den Heuvel-Eibrink Jeroen A. W. M. van der Laak Marta Fiocco Annelies M. C. Mavinkurve-Groothuis Ronald R. de Krijger |
author_sort |
Jikke J. Rutgers |
title |
Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach |
title_short |
Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach |
title_full |
Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach |
title_fullStr |
Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach |
title_full_unstemmed |
Interobserver variability between experienced and inexperienced observers in the histopathological analysis of Wilms tumors: a pilot study for future algorithmic approach |
title_sort |
interobserver variability between experienced and inexperienced observers in the histopathological analysis of wilms tumors: a pilot study for future algorithmic approach |
publisher |
BMC |
series |
Diagnostic Pathology |
issn |
1746-1596 |
publishDate |
2021-08-01 |
description |
Abstract Background Histopathological classification of Wilms tumors determines treatment regimen. Machine learning has been shown to contribute to histopathological classification in various malignancies but requires large numbers of manually annotated images and thus specific pathological knowledge. This study aimed to assess whether trained, inexperienced observers could contribute to reliable annotation of Wilms tumor components for classification performed by machine learning. Methods Four inexperienced observers (medical students) were trained in histopathology of normal kidneys and Wilms tumors by an experienced observer (pediatric pathologist). Twenty randomly selected scanned Wilms tumor-slides (from n = 1472 slides) were annotated, and annotations were independently classified by both the inexperienced observers and two experienced pediatric pathologists. Agreement between the six observers and for each tissue element was measured using kappa statistics (κ). Results Pairwise interobserver agreement between all inexperienced and experienced observers was high (range: 0.845–0.950). The interobserver variability for the different histological elements, including all vital tumor components and therapy-related effects, showed high values for all κ-coefficients (> 0.827). Conclusions Inexperienced observers can be trained to recognize specific histopathological tumor and tissue elements with high interobserver agreement with experienced observers. Nevertheless, supervision by experienced pathologists remains necessary. Results of this study can be used to facilitate more rapid progress for supervised machine learning-based algorithm development in pediatric pathology and beyond. |
topic |
Wilms tumor Interobserver variability Machine learning Histopathology Classification AI (artificial intelligence) |
url |
https://doi.org/10.1186/s13000-021-01136-w |
work_keys_str_mv |
AT jikkejrutgers interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT tessabanki interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT anandavanderkamp interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT tomasjwaterlander interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT marijnascheijdevermeulen interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT marrymvandenheuveleibrink interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT jeroenawmvanderlaak interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT martafiocco interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT anneliesmcmavinkurvegroothuis interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach AT ronaldrdekrijger interobservervariabilitybetweenexperiencedandinexperiencedobserversinthehistopathologicalanalysisofwilmstumorsapilotstudyforfuturealgorithmicapproach |
_version_ |
1721199449245483008 |