Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data
Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity o...
Main Authors: | , , , , |
---|---|
Format: | Article |
Language: | English |
Published: |
Frontiers Media S.A.
2021-02-01
|
Series: | Frontiers in Cellular and Infection Microbiology |
Subjects: | |
Online Access: | https://www.frontiersin.org/articles/10.3389/fcimb.2021.610348/full |
id |
doaj-38eb5253b32c4e2a9cbbe73644fa5c97 |
---|---|
record_format |
Article |
spelling |
doaj-38eb5253b32c4e2a9cbbe73644fa5c972021-02-15T04:16:21ZengFrontiers Media S.A.Frontiers in Cellular and Infection Microbiology2235-29882021-02-011110.3389/fcimb.2021.610348610348Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing DataLukas Lüftinger0Lukas Lüftinger1Peter Májek2Stephan Beisken3Thomas Rattei4Andreas E. Posch5Ares Genetics GmbH, Vienna, AustriaDivision of Computational Systems Biology, Department of Microbiology and Ecosystem Science, University of Vienna, Vienna, AustriaAres Genetics GmbH, Vienna, AustriaAres Genetics GmbH, Vienna, AustriaDivision of Computational Systems Biology, Department of Microbiology and Ecosystem Science, University of Vienna, Vienna, AustriaAres Genetics GmbH, Vienna, AustriaAntimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST.https://www.frontiersin.org/articles/10.3389/fcimb.2021.610348/fullmachine learninggenomicsantimicrobial resistanceantibioticswhole genome sequencing (WGS) |
collection |
DOAJ |
language |
English |
format |
Article |
sources |
DOAJ |
author |
Lukas Lüftinger Lukas Lüftinger Peter Májek Stephan Beisken Thomas Rattei Andreas E. Posch |
spellingShingle |
Lukas Lüftinger Lukas Lüftinger Peter Májek Stephan Beisken Thomas Rattei Andreas E. Posch Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data Frontiers in Cellular and Infection Microbiology machine learning genomics antimicrobial resistance antibiotics whole genome sequencing (WGS) |
author_facet |
Lukas Lüftinger Lukas Lüftinger Peter Májek Stephan Beisken Thomas Rattei Andreas E. Posch |
author_sort |
Lukas Lüftinger |
title |
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_short |
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_full |
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_fullStr |
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_full_unstemmed |
Learning From Limited Data: Towards Best Practice Techniques for Antimicrobial Resistance Prediction From Whole Genome Sequencing Data |
title_sort |
learning from limited data: towards best practice techniques for antimicrobial resistance prediction from whole genome sequencing data |
publisher |
Frontiers Media S.A. |
series |
Frontiers in Cellular and Infection Microbiology |
issn |
2235-2988 |
publishDate |
2021-02-01 |
description |
Antimicrobial resistance prediction from whole genome sequencing data (WGS) is an emerging application of machine learning, promising to improve antimicrobial resistance surveillance and outbreak monitoring. Despite significant reductions in sequencing cost, the availability and sampling diversity of WGS data with matched antimicrobial susceptibility testing (AST) profiles required for training of WGS-AST prediction models remains limited. Best practice machine learning techniques are required to ensure trained models generalize to independent data for optimal predictive performance. Limited data restricts the choice of machine learning training and evaluation methods and can result in overestimation of model performance. We demonstrate that the widely used random k-fold cross-validation method is ill-suited for application to small bacterial genomics datasets and offer an alternative cross-validation method based on genomic distance. We benchmarked three machine learning architectures previously applied to the WGS-AST problem on a set of 8,704 genome assemblies from five clinically relevant pathogens across 77 species-compound combinations collated from public databases. We show that individual models can be effectively ensembled to improve model performance. By combining models via stacked generalization with cross-validation, a model ensembling technique suitable for small datasets, we improved average sensitivity and specificity of individual models by 1.77% and 3.20%, respectively. Furthermore, stacked models exhibited improved robustness and were thus less prone to outlier performance drops than individual component models. In this study, we highlight best practice techniques for antimicrobial resistance prediction from WGS data and introduce the combination of genome distance aware cross-validation and stacked generalization for robust and accurate WGS-AST. |
topic |
machine learning genomics antimicrobial resistance antibiotics whole genome sequencing (WGS) |
url |
https://www.frontiersin.org/articles/10.3389/fcimb.2021.610348/full |
work_keys_str_mv |
AT lukasluftinger learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT lukasluftinger learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT petermajek learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT stephanbeisken learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT thomasrattei learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata AT andreaseposch learningfromlimiteddatatowardsbestpracticetechniquesforantimicrobialresistancepredictionfromwholegenomesequencingdata |
_version_ |
1724269170316541952 |