An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities

Abstract Despite the advent of whole genome metagenomics, targeted approaches (such as 16S rRNA gene amplicon sequencing) continue to be valuable for determining the microbial composition of samples. Amplicon microbiome sequencing can be performed on clinical samples from a normally sterile site to...

Full description

Bibliographic Details
Main Authors: Denise M. O’Sullivan, Ronan M. Doyle, Sasithon Temisak, Nicholas Redshaw, Alexandra S. Whale, Grace Logan, Jiabin Huang, Nicole Fischer, Gregory C. A. Amos, Mark D. Preston, Julian R. Marchesi, Josef Wagner, Julian Parkhill, Yair Motro, Hubert Denise, Robert D. Finn, Kathryn A. Harris, Gemma L. Kay, Justin O’Grady, Emma Ransom-Jones, Huihai Wu, Emma Laing, David J. Studholme, Ernest Diez Benavente, Jody Phelan, Taane G. Clark, Jacob Moran-Gilad, Jim F. Huggett
Format: Article
Language:English
Published: Nature Publishing Group 2021-05-01
Series:Scientific Reports
Online Access:https://doi.org/10.1038/s41598-021-89881-2
id doaj-8468c62cda324455a0cd7a1fb1c847ee
record_format Article
collection DOAJ
language English
format Article
sources DOAJ
author Denise M. O’Sullivan
Ronan M. Doyle
Sasithon Temisak
Nicholas Redshaw
Alexandra S. Whale
Grace Logan
Jiabin Huang
Nicole Fischer
Gregory C. A. Amos
Mark D. Preston
Julian R. Marchesi
Josef Wagner
Julian Parkhill
Yair Motro
Hubert Denise
Robert D. Finn
Kathryn A. Harris
Gemma L. Kay
Justin O’Grady
Emma Ransom-Jones
Huihai Wu
Emma Laing
David J. Studholme
Ernest Diez Benavente
Jody Phelan
Taane G. Clark
Jacob Moran-Gilad
Jim F. Huggett
spellingShingle Denise M. O’Sullivan
Ronan M. Doyle
Sasithon Temisak
Nicholas Redshaw
Alexandra S. Whale
Grace Logan
Jiabin Huang
Nicole Fischer
Gregory C. A. Amos
Mark D. Preston
Julian R. Marchesi
Josef Wagner
Julian Parkhill
Yair Motro
Hubert Denise
Robert D. Finn
Kathryn A. Harris
Gemma L. Kay
Justin O’Grady
Emma Ransom-Jones
Huihai Wu
Emma Laing
David J. Studholme
Ernest Diez Benavente
Jody Phelan
Taane G. Clark
Jacob Moran-Gilad
Jim F. Huggett
An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
Scientific Reports
author_facet Denise M. O’Sullivan
Ronan M. Doyle
Sasithon Temisak
Nicholas Redshaw
Alexandra S. Whale
Grace Logan
Jiabin Huang
Nicole Fischer
Gregory C. A. Amos
Mark D. Preston
Julian R. Marchesi
Josef Wagner
Julian Parkhill
Yair Motro
Hubert Denise
Robert D. Finn
Kathryn A. Harris
Gemma L. Kay
Justin O’Grady
Emma Ransom-Jones
Huihai Wu
Emma Laing
David J. Studholme
Ernest Diez Benavente
Jody Phelan
Taane G. Clark
Jacob Moran-Gilad
Jim F. Huggett
author_sort Denise M. O’Sullivan
title An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
title_short An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
title_full An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
title_fullStr An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
title_full_unstemmed An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
title_sort inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communities
publisher Nature Publishing Group
series Scientific Reports
issn 2045-2322
publishDate 2021-05-01
description Abstract Despite the advent of whole genome metagenomics, targeted approaches (such as 16S rRNA gene amplicon sequencing) continue to be valuable for determining the microbial composition of samples. Amplicon microbiome sequencing can be performed on clinical samples from a normally sterile site to determine the aetiology of an infection (usually single pathogen identification) or samples from more complex niches such as human mucosa or environmental samples where multiple microorganisms need to be identified. The methodologies are frequently applied to determine both presence of micro-organisms and their quantity or relative abundance. There are a number of technical steps required to perform microbial community profiling, many of which may have appreciable precision and bias that impacts final results. In order for these methods to be applied with the greatest accuracy, comparative studies across different laboratories are warranted. In this study we explored the impact of the bioinformatic approaches taken in different laboratories on microbiome assessment using 16S rRNA gene amplicon sequencing results. Data were generated from two mock microbial community samples which were amplified using primer sets spanning five different variable regions of 16S rRNA genes. The PCR-sequencing analysis included three technical repeats of the process to determine the repeatability of their methods. Thirteen laboratories participated in the study, and each analysed the same FASTQ files using their choice of pipeline. This study captured the methods used and the resulting sequence annotation and relative abundance output from bioinformatic analyses. Results were compared to digital PCR assessment of the absolute abundance of each target representing each organism in the mock microbial community samples and also to analyses of shotgun metagenome sequence data. This ring trial demonstrates that the choice of bioinformatic analysis pipeline alone can result in different estimations of the composition of the microbiome when using 16S rRNA gene amplicon sequencing data. The study observed differences in terms of both presence and abundance of organisms and provides a resource for ensuring reproducible pipeline development and application. The observed differences were especially prevalent when using custom databases and applying high stringency operational taxonomic unit (OTU) cut-off limits. In order to apply sequencing approaches with greater accuracy, the impact of different analytical steps needs to be clearly delineated and solutions devised to harmonise microbiome analysis results.
url https://doi.org/10.1038/s41598-021-89881-2
work_keys_str_mv AT denisemosullivan aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT ronanmdoyle aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT sasithontemisak aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT nicholasredshaw aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT alexandraswhale aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gracelogan aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jiabinhuang aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT nicolefischer aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gregorycaamos aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT markdpreston aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT julianrmarchesi aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT josefwagner aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT julianparkhill aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT yairmotro aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT hubertdenise aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT robertdfinn aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT kathrynaharris aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gemmalkay aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT justinogrady aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT emmaransomjones aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT huihaiwu aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT emmalaing aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT davidjstudholme aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT ernestdiezbenavente aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jodyphelan aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT taanegclark aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jacobmorangilad aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jimfhuggett aninterlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT denisemosullivan interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT ronanmdoyle interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT sasithontemisak interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT nicholasredshaw interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT alexandraswhale interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gracelogan interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jiabinhuang interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT nicolefischer interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gregorycaamos interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT markdpreston interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT julianrmarchesi interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT josefwagner interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT julianparkhill interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT yairmotro interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT hubertdenise interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT robertdfinn interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT kathrynaharris interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT gemmalkay interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT justinogrady interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT emmaransomjones interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT huihaiwu interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT emmalaing interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT davidjstudholme interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT ernestdiezbenavente interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jodyphelan interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT taanegclark interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jacobmorangilad interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
AT jimfhuggett interlaboratorystudytoinvestigatetheimpactofthebioinformaticscomponentonmicrobiomeanalysisusingmockcommunities
_version_ 1721429589755953152
spelling doaj-8468c62cda324455a0cd7a1fb1c847ee2021-05-23T11:31:31ZengNature Publishing GroupScientific Reports2045-23222021-05-0111111410.1038/s41598-021-89881-2An inter-laboratory study to investigate the impact of the bioinformatics component on microbiome analysis using mock communitiesDenise M. O’Sullivan0Ronan M. Doyle1Sasithon Temisak2Nicholas Redshaw3Alexandra S. Whale4Grace Logan5Jiabin Huang6Nicole Fischer7Gregory C. A. Amos8Mark D. Preston9Julian R. Marchesi10Josef Wagner11Julian Parkhill12Yair Motro13Hubert Denise14Robert D. Finn15Kathryn A. Harris16Gemma L. Kay17Justin O’Grady18Emma Ransom-Jones19Huihai Wu20Emma Laing21David J. Studholme22Ernest Diez Benavente23Jody Phelan24Taane G. Clark25Jacob Moran-Gilad26Jim F. Huggett27Molecular Biology, National Measurement Laboratory, LGCDepartment of Microbiology, Virology and Infection Control, Great Ormond Street Hospital for Children NHS TrustMolecular Biology, National Measurement Laboratory, LGCMolecular Biology, National Measurement Laboratory, LGCMolecular Biology, National Measurement Laboratory, LGCDepartment of Infection, Immunity and Inflammation, UCL Great Ormond Street Institute of Child Health and Reubens Centre of Paediatric Virology and MetagenomicsInstitute of Medical Microbiology, Virology and Hygiene, University Medical Center Hamburg-Eppendorf, UKEInstitute of Medical Microbiology, Virology and Hygiene, University Medical Center Hamburg-Eppendorf, UKEDepartment of Bacteriology, TDI, National Institute for Biological Standards and ControlDepartment of Bacteriology, TDI, National Institute for Biological Standards and ControlSchool of Biosciences, Cardiff UniversityPathogens and Microbes, Wellcome Sanger Institute, Wellcome Genome CampusPathogens and Microbes, Wellcome Sanger Institute, Wellcome Genome CampusDepartment of Health System Management, School of Public Health, Faculty of Health Sciences, Ben-Gurion University of the NegevEuropean Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome CampusEuropean Molecular Biology Laboratory, European Bioinformatics Institute (EMBL-EBI), Wellcome Trust Genome CampusDepartment of Microbiology, Virology and Infection Control, Great Ormond Street Hospital for Children NHS TrustMedical Microbiology Research Laboratory, Bob Champion Research and Educational Building, University of East AngliaMedical Microbiology Research Laboratory, Bob Champion Research and Educational Building, University of East AngliaDepartment of Biological and Geographical Sciences, School of Applied Sciences, University of HuddersfieldSchool of Biosciences, University of SurreySchool of Biosciences, University of SurreyBiosciences, University of ExeterFaculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical MedicineFaculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical MedicineFaculty of Infectious and Tropical Diseases, London School of Hygiene and Tropical MedicineDepartment of Health System Management, School of Public Health, Faculty of Health Sciences, Ben-Gurion University of the NegevMolecular Biology, National Measurement Laboratory, LGCAbstract Despite the advent of whole genome metagenomics, targeted approaches (such as 16S rRNA gene amplicon sequencing) continue to be valuable for determining the microbial composition of samples. Amplicon microbiome sequencing can be performed on clinical samples from a normally sterile site to determine the aetiology of an infection (usually single pathogen identification) or samples from more complex niches such as human mucosa or environmental samples where multiple microorganisms need to be identified. The methodologies are frequently applied to determine both presence of micro-organisms and their quantity or relative abundance. There are a number of technical steps required to perform microbial community profiling, many of which may have appreciable precision and bias that impacts final results. In order for these methods to be applied with the greatest accuracy, comparative studies across different laboratories are warranted. In this study we explored the impact of the bioinformatic approaches taken in different laboratories on microbiome assessment using 16S rRNA gene amplicon sequencing results. Data were generated from two mock microbial community samples which were amplified using primer sets spanning five different variable regions of 16S rRNA genes. The PCR-sequencing analysis included three technical repeats of the process to determine the repeatability of their methods. Thirteen laboratories participated in the study, and each analysed the same FASTQ files using their choice of pipeline. This study captured the methods used and the resulting sequence annotation and relative abundance output from bioinformatic analyses. Results were compared to digital PCR assessment of the absolute abundance of each target representing each organism in the mock microbial community samples and also to analyses of shotgun metagenome sequence data. This ring trial demonstrates that the choice of bioinformatic analysis pipeline alone can result in different estimations of the composition of the microbiome when using 16S rRNA gene amplicon sequencing data. The study observed differences in terms of both presence and abundance of organisms and provides a resource for ensuring reproducible pipeline development and application. The observed differences were especially prevalent when using custom databases and applying high stringency operational taxonomic unit (OTU) cut-off limits. In order to apply sequencing approaches with greater accuracy, the impact of different analytical steps needs to be clearly delineated and solutions devised to harmonise microbiome analysis results.https://doi.org/10.1038/s41598-021-89881-2