Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey
Aim: Male patients with anorectal malformations (ARM) are classified according to presence and level of the recto-urinary fistula. This is traditionally established by a preoperative high-pressure distal colostogram that may be variably interpreted by different surgeons. The aim of this study was to...
Main Authors: | , , , , , , , , , , , , , , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2020-09-01
|
Series: | Frontiers in Pediatrics |
Subjects: | |
Online Access: | https://www.frontiersin.org/article/10.3389/fped.2020.00571/full |
id |
doaj-3d85f2a9fb5441599c74234dedf12c60 |
---|---|
record_format |
Article |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Paola Midrio Iris A. L. M. van Rooij Giulia Brisighelli Giulia Brisighelli Aracelli Garcia Maria Fanjul Paul Broens Barbara D. Iacobelli Carlos Giné Gabriele Lisi Cornelius E. J. Sloots Francesco Fascetti Leon Anna Morandi Herjan van der Steeg Stefan Giuliani Sabine Grasshoff-Derr Martin Lacher Ivo de Blaauw Ekkehart Jenetzky Ekkehart Jenetzky |
spellingShingle |
Paola Midrio Iris A. L. M. van Rooij Giulia Brisighelli Giulia Brisighelli Aracelli Garcia Maria Fanjul Paul Broens Barbara D. Iacobelli Carlos Giné Gabriele Lisi Cornelius E. J. Sloots Francesco Fascetti Leon Anna Morandi Herjan van der Steeg Stefan Giuliani Sabine Grasshoff-Derr Martin Lacher Ivo de Blaauw Ekkehart Jenetzky Ekkehart Jenetzky Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey Frontiers in Pediatrics anorectal malformation urinary fistula colostogram surgery ARM-net multirater agreement |
author_facet |
Paola Midrio Iris A. L. M. van Rooij Giulia Brisighelli Giulia Brisighelli Aracelli Garcia Maria Fanjul Paul Broens Barbara D. Iacobelli Carlos Giné Gabriele Lisi Cornelius E. J. Sloots Francesco Fascetti Leon Anna Morandi Herjan van der Steeg Stefan Giuliani Sabine Grasshoff-Derr Martin Lacher Ivo de Blaauw Ekkehart Jenetzky Ekkehart Jenetzky |
author_sort |
Paola Midrio |
title |
Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey |
title_short |
Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey |
title_full |
Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey |
title_fullStr |
Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey |
title_full_unstemmed |
Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium Survey |
title_sort |
inter- and intraobserver variation in the assessment of preoperative colostograms in male anorectal malformations: an arm-net consortium survey |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Pediatrics |
issn |
2296-2360 |
publishDate |
2020-09-01 |
description |
Aim: Male patients with anorectal malformations (ARM) are classified according to presence and level of the recto-urinary fistula. This is traditionally established by a preoperative high-pressure distal colostogram that may be variably interpreted by different surgeons. The aim of this study was to evaluate the inter- and intraobserver variation in the assessment by pediatric surgeons of preoperative colostograms with respect to the level of the recto-urinary fistula.Materials and Methods: Sixteen pediatric surgeons from 14 European centers belonging to the ARM-Net Consortium twice scored 130 images of distal colostograms taken in sagittal projection at a median age of 66 days of life (range: 4–1,106 days). Surgeons were asked to classify the fistula in bulbar, prostatic, bladder-neck, no fistula, and “unclear anatomy” example. Their assessments were compared with the intraoperative findings (kappa) for two scoring rounds with an interval of 6 months (intraobserver variation). Agreement among the surgeons' scores (interobserver variation) was also calculated using Krippendorff's alpha. A kappa over 0.75 is considered excellent, between 0.40 and 0.75 fair to good, and below 0.40 poor. Surgeons were asked to score the images in “poor” and “good” quality and to provide their years of experience in ARM treatment.Results: Agreement between the image-based rating of surgeons and the intraoperative findings ranges from 0.06 to 0.45 (mean 0.31). Interobserver variation is higher (Krippendorff's alpha between 0.40 and 0.45). Years of experience in ARM treatment does not seem to influence the scoring. The mean intraobserver variation between the two rounds is 0.64. Overall, the quality of the images is considered poor. Images categorized as having a good quality result in a statistically significant higher kappa (mean: 0.36 and 0.37 in the first and second round, respectively) than in the group of bad-quality images (mean: 0.25 and 0.23, respectively).Conclusions: There is poor agreement among experienced pediatric colorectal surgeons on preoperative colostograms. Techniques and analyses of images need to be improved in order to generate a homogeneous series of patients and make comparison of outcomes reliable. |
topic |
anorectal malformation urinary fistula colostogram surgery ARM-net multirater agreement |
url |
https://www.frontiersin.org/article/10.3389/fped.2020.00571/full |
work_keys_str_mv |
AT paolamidrio interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT irisalmvanrooij interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT giuliabrisighelli interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT giuliabrisighelli interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT aracelligarcia interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT mariafanjul interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT paulbroens interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT barbaradiacobelli interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT carlosgine interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT gabrielelisi interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT corneliusejsloots interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT francescofascettileon interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT annamorandi interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT herjanvandersteeg interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT stefangiuliani interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT sabinegrasshoffderr interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT martinlacher interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT ivodeblaauw interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT ekkehartjenetzky interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey AT ekkehartjenetzky interandintraobservervariationintheassessmentofpreoperativecolostogramsinmaleanorectalmalformationsanarmnetconsortiumsurvey |
_version_ |
1724607236293001216 |
spelling |
doaj-3d85f2a9fb5441599c74234dedf12c602020-11-25T03:23:10ZengFrontiers Media S.A.Frontiers in Pediatrics2296-23602020-09-01810.3389/fped.2020.00571572541Inter- and Intraobserver Variation in the Assessment of Preoperative Colostograms in Male Anorectal Malformations: An ARM-Net Consortium SurveyPaola Midrio0Iris A. L. M. van Rooij1Giulia Brisighelli2Giulia Brisighelli3Aracelli Garcia4Maria Fanjul5Paul Broens6Barbara D. Iacobelli7Carlos Giné8Gabriele Lisi9Cornelius E. J. Sloots10Francesco Fascetti Leon11Anna Morandi12Herjan van der Steeg13Stefan Giuliani14Sabine Grasshoff-Derr15Martin Lacher16Ivo de Blaauw17Ekkehart Jenetzky18Ekkehart Jenetzky19Pediatric Surgery Unit, Cà Foncello Hospital, Treviso, ItalyDepartment of Health Evidence, Radboud Institute for Health Sciences, Nijmegen, NetherlandsDepartment of Pediatric Surgery, Fondazione IRCCS Ca' Granda- Ospedale Maggiore Policlinico, Milan, ItalyDepartment of Pediatric Surgery, Chris Hani Baragwanath Hospital and University of Witwatersrand, Johannesburg, South AfricaPediatric Surgery Unit, Doce de Octubre Universitary Hospital, Madrid, SpainPediatric Surgery Unit, Gregorio Marañón Universitary Hospital, Madrid, SpainDivision of Pediatric Surgery, Department of Surgery, University Medical Center Groningen, University of Groningen, Groningen, NetherlandsDepartment of Medical and Surgical Neonatology, Bambino Gesù Childrens Hospital, Rome, ItalyDepartment of Pediatric Surgery, Hospital Universitary Vall d'Hebron, Barcelona, Spain0Pediatric Surgery Unit, Department of Aging Science, University “G. d'Annunzio” of Chieti-Pescara, Chieti, Italy1Department of Pediatric Surgery, Erasmus Medical Center (MC)–Sophia Children's Hospital, Rotterdam, Netherlands2Pediatric Surgery Unit, University Hospital of Padua, Padua, ItalyDepartment of Pediatric Surgery, Fondazione IRCCS Ca' Granda- Ospedale Maggiore Policlinico, Milan, Italy3Division of Pediatric Surgery, Department of Surgery, Radboudumc-Amalia Children's Hospital, Nijmegen, Netherlands4Department of Pediatric Surgery, St. George's University Hospitals NHS Foundation Trust, London, United Kingdom5Pediatric Surgery Unit, Buergerhospital and Clementine Kinderhospital, Frankfurt, Germany6Department of Pediatric Surgery, University of Leipzig, Leipzig, Germany3Division of Pediatric Surgery, Department of Surgery, Radboudumc-Amalia Children's Hospital, Nijmegen, Netherlands7Department of Medicine, Faculty of Health, Witten/Herdecke University, Witten, Germany8Department of Child and Adolescent Psychiatry and Psychotherapy, University Medical Center of Johannes Gutenberg-University, Mainz, GermanyAim: Male patients with anorectal malformations (ARM) are classified according to presence and level of the recto-urinary fistula. This is traditionally established by a preoperative high-pressure distal colostogram that may be variably interpreted by different surgeons. The aim of this study was to evaluate the inter- and intraobserver variation in the assessment by pediatric surgeons of preoperative colostograms with respect to the level of the recto-urinary fistula.Materials and Methods: Sixteen pediatric surgeons from 14 European centers belonging to the ARM-Net Consortium twice scored 130 images of distal colostograms taken in sagittal projection at a median age of 66 days of life (range: 4–1,106 days). Surgeons were asked to classify the fistula in bulbar, prostatic, bladder-neck, no fistula, and “unclear anatomy” example. Their assessments were compared with the intraoperative findings (kappa) for two scoring rounds with an interval of 6 months (intraobserver variation). Agreement among the surgeons' scores (interobserver variation) was also calculated using Krippendorff's alpha. A kappa over 0.75 is considered excellent, between 0.40 and 0.75 fair to good, and below 0.40 poor. Surgeons were asked to score the images in “poor” and “good” quality and to provide their years of experience in ARM treatment.Results: Agreement between the image-based rating of surgeons and the intraoperative findings ranges from 0.06 to 0.45 (mean 0.31). Interobserver variation is higher (Krippendorff's alpha between 0.40 and 0.45). Years of experience in ARM treatment does not seem to influence the scoring. The mean intraobserver variation between the two rounds is 0.64. Overall, the quality of the images is considered poor. Images categorized as having a good quality result in a statistically significant higher kappa (mean: 0.36 and 0.37 in the first and second round, respectively) than in the group of bad-quality images (mean: 0.25 and 0.23, respectively).Conclusions: There is poor agreement among experienced pediatric colorectal surgeons on preoperative colostograms. Techniques and analyses of images need to be improved in order to generate a homogeneous series of patients and make comparison of outcomes reliable.https://www.frontiersin.org/article/10.3389/fped.2020.00571/fullanorectal malformationurinary fistulacolostogramsurgeryARM-netmultirater agreement |