Systematic review searches must be systematic, comprehensive, and transparent: a critique of Perman et al

• Main Authors: Devon Greyson, Ellen Rafferty, Linda Slater, Noni MacDonald, Julie A. Bettinger, Ève Dubé, Shannon E. MacDonald
• Format: Article
• Language: English
• Published: BMC 2019-02-01
• Series: BMC Public Health
• Subjects: Systematic review; Narrative review; Search methodology; School; Vaccination; Immunization
• Online Access: http://link.springer.com/article/10.1186/s12889-018-6275-y

Abstract A high quality systematic review search has three core attributes; it is systematic, comprehensive, and transparent. The current over-emphasis on the primacy of systematic reviews over other forms of literature review in health research, however, runs the risk of encouraging publication of reviews whose searches do not meet these three criteria under the guise of being systematic reviews. This correspondence comes in response to Perman S, Turner S, Ramsay AIG, Baim-Lance A, Utley M, Fulop NJ. School-based vaccination programmes: a systematic review of the evidence on organization and delivery in high income countries. 2017; BMC Public Health 17:252, which we assert did not meet these three important quality criteria for systematic reviews, thereby leading to potentially unreliable conclusions. Our aims herein are to emphasize the importance of maintaining a high degree of rigour in the conduct and publication of systematic reviews that may be used by clinicians and policy-makers to guide or alter practice or policy, and to highlight and discuss key evidence omitted in the published review in order to contextualize the findings for readers. By consulting a research librarian, we identified limitations in the search terms, the number and type of databases, and the screening methods used by Perman et al. Using a revised Ovid MEDLINE search strategy, we identified an additional 1016 records in that source alone, and highlighted relevant literature on the organization and delivery of school-based immunization program that was omitted as a result. We argue that a number of the literature gaps noted by Perman et al. may well be addressed by existing literature found through a more systematic and comprehensive search and screening strategy. We commend both the journal and the authors, however, for their transparency in supplying information about the search strategy and providing open access to peer reviewer and editor’s comments, which enabled us to understand the reasons for the limitations of that review.