Reviews: Rapid! Rapid! Rapid! …and systematic

Abstract

“Although time is a reality, lack of time is not lack of reality” - HOJES Systematic reviews Systematic reviews are scientific investigations, with pre-planned methods and an assembly of original studies as their “subjects” [1, 2]. They synthesize the results of multiple primary investigations by using strategies that limit bias and random error [1, 2]. Systematic reviews are transparent about how studies were identified and which were included or excluded, the risk of bias assessment, and the methods to summarize data and assess the certainty in the evidence. Standards for the conduct of systematic reviews have been made available by the Cochrane Collaboration and other organizations. If systematic reviews are done well, e.g., by adhering to conduct (e.g., Cochrane Handbook for Systematic Reviews of Interventions) and reporting (e.g., PRISMA Statement) best practice standards, it is not sensible to question the value of systematic reviews as a source of information for shaping decision making [3, 4]. This methodology of systematic reviews—although laid out three or more decades ago—is continuously and rapidly updated by scientists specializing in research synthesis. Now, Systematic Reviews is publishing a series of articles including methods and examples of accelerating approaches to conducting literature reviews. As a rule of thumb, rapid systematic reviews should be conducted in less than 8 weeks, including protocol publication. On the whole, this is a saving of about 75% in terms of time compared to what most researchers would propose as standard timeline for systematic reviews. Examples will highlight how health policy decisions can be influenced when a rapid review methodology is used. The challenge of traditional systematic reviews Findings from a single randomized trial are often rapidly challenged by succeeding studies, and rigorous systematic reviews help approximate “true evidence” and estimates in effects [5]. High-quality systematic reviews are used more often and are considered more trustworthy by health professionals in terms of relevance to clinical practice than other types of designs [6]. However, conducting and adhering to the standards of traditional systematic reviews can be time consuming. The reason for that lies in the rigorous approach to methods ensuring that the best available evidence is identified, assessed, and synthesized. But those demanding evidence syntheses for decision making are increasingly living in faster paced times, influenced by innovative interventions and technology that accelerate communication and interaction. Decision makers often do not appreciate the intricacies of research methods and the time needed to comply with the task. The argument that transparency is ensured by completing a traditional systematic reviews is often not convincing enough. But there are other reasons. The rationale for rapid—systematic—reviews The concern regarding a timely decision on health care and policies is the driving force for rapid reviews. In fact, decision making should not be delayed in most situations and cannot be delayed in some. In the face of a tragic Ebola epidemic, we are reminded of how rapidly answers are required. To base answers on the best available evidence, this evidence must be synthesized without undue delays. While typical systematic reviews can take years to complete (one of the author was involved in a systematic review that took 12 years to complete), rapid reviews are required when facing such dramatic situations. Prior to the Ebola epidemic, the fear of avian influenza prompted the World Health Organization to offer rapid guidelines that were supported by a rapid review methodology. From guideline panel formation to completion of the recommendations, only about 12 weeks passed [7]. Another recent rapid systematic review was commissioned to inform decision making with regard to the safety of two drugs, bevacizumab (Avastin) and ranibizumab (Lucentis), widely used to stabilize vision in patients with neovascular age-related macular degeneration. From team formation to completion of the systematic review, only about 8 weeks passed. Another 8 weeks was necessary to complete the publication process and publish it as a Cochrane review [8]. These and other experiences, such as the ones presented in this series, show that rapid evidence synthesis can be done to support decisions ranging from clinical to health policy. The pitfalls of rapid reviews Should rapidity be considered as a key risk factor for poor, overly simplistic, or frankly misconducted systematic reviews? We do not think so. Rapidity by itself is not a predictor of the quality of a systematic review. The same amount and quality of work can be completed in a shorter or longer time, although sometimes saving in time might be accompanied by compromising in conduct. This includes missing important evidence and errors in the assessment or synthesis of the evidence. Systematic review authors and users of systematic reviews must, however, resist the pressures of shortcuts when they suggest bias [9]. The quality of systematic reviews and meta-analyses should be evaluated irrespective of their speed. What rapid—systematic—reviews must do Apart from time, what makes rapid systematic reviews different from traditional systematic reviews? Not the amount of work. Rapid reviews must remain systematic by adhering to the core principles of systematic reviews that avoid bias in the inclusions, assessment, and synthesis of studies. The methods sections will be of greater importance as deviations from traditional systematic review methods should be laid out clearly. Thus, contrary to what the label “rapid” may imply, transparency in the description of the methods used will become more important; rapidity is not a justification for brevity, and rapidity should not be confused with brevity. One approach to increasing transparency will be highlighting where the PRISMA criteria were omitted or modified. Rapid reviews can remain systematic if the core principles are adhered to, and that should be reflected in the methods and title. A note on resources An important issue that might differentiate rapid systematic reviews from traditional systematic reviews is the more marked need to support production across the review’s lifecycle, from early question generation and method planning to development of the manuscript, followed by the release of user-friendly communication tools (e.g., summary of finding tables). The speed of review conduct can be directly correlated with the availability of resources, both human and financial (which, in turn, may ensure human resources). Adequate planning requires lining up all activities against review deliverables and timelines and harmonizing the required expertise in a more streamlined fashion. Since rapid systematic reviews often require reaching a consensus about disputed evidence more quickly, they might involve stakeholders with different backgrounds earlier. This aspect also has resource implications, since review drafts will circulate quickly between authors and require closer attention to each round of revision. When rapid reviews include many studies, it might be expedient to increase the number of reviewers involved. However, increasing the team size has costs, too: the possibility of greater interindividual reliability on study inclusion and data abstraction must be accepted. It is important to ensure that all reviewers are well trained in systematic review methods and ensure attention to review execution at each step even under pressure. In this way, producing rapid systematic reviews that do not fall short in terms of the applied methodological rigor remains a reality. Summary and terminology In summary, if there is no compromise of the validity of a review, then reviews should be done rapidly. This would mean that one does not accept shortcuts in terms of methods for review conduct. Thus, the term rapid review is a possible misnomer and conceptually wrong (in the authors’ view). Velocity does not have to impact transparency and appropriate methods. Rapid reviewers must do their utmost to adhere to guidelines for review conduct and reporting. As evidence is only slowly emerging as to which steps in the systematic review process may be altered by increased speed and will require more examples such as the ones described by the authors of this series in Systematic Reviews, transparency is key. Rapid reviews that are not systematic bear the risks of any other narrative review or poorly conducted systematic reviews [10].