Random forest machine learning method outperforms prehospital National Early Warning Score for predicting one-day mortality: A retrospective study

• Main Authors: Jussi Pirneskoski, Joonas Tamminen, Antti Kallonen, Jouni Nurmi, Markku Kuisma, Klaus T. Olkkola, Sanna Hoppu
• Format: Article
• Language: English
• Published: Elsevier 2020-12-01
• Series: Resuscitation Plus
• Subjects: Emergency medical services; Prehospital; Cardiac arrest prevention; Early warning score; National Early Warning Score; NEWS
• Online Access: http://www.sciencedirect.com/science/article/pii/S2666520420300461

Aim of the study: The National Early Warning Score (NEWS) is a validated method for predicting clinical deterioration in hospital wards, but its performance in prehospital settings remains controversial. Modern machine learning models may outperform traditional statistical analyses for predicting short-term mortality. Thus, we aimed to compare the mortality prediction accuracy of NEWS and random forest machine learning using prehospital vital signs. Methods: In this retrospective study, all electronic ambulance mission reports between 2008 and 2015 in a single EMS system were collected. Adult patients (≥ 18 years) were included in the analysis. Random forest models with and without blood glucose were compared to the traditional NEWS for predicting one-day mortality. A ten-fold cross-validation method was applied to train and validate the random forest models. Results: A total of 26,458 patients were included in the study of whom 278 (1.0%) died within one day of ambulance mission. The area under the receiver operating characteristic curve for one-day mortality was 0.836 (95% CI, 0.810−0.860) for NEWS, 0.858 (95% CI, 0.832−0.883) for a random forest trained with NEWS variables only and 0.868 (0.843−0.892) for a random forest trained with NEWS variables and blood glucose. Conclusion: A random forest algorithm trained with NEWS variables was superior to traditional NEWS for predicting one-day mortality in adult prehospital patients, although the risk of selection bias must be acknowledged. The inclusion of blood glucose in the model further improved its predictive performance.