Systematic Reviews and other evidence synthesis projects

"A systematic review (SR) attempts to collate all empirical evidence that fits pre-specified eligibility criteria in order to answer a specific research question.  It  uses explicit, systematic methods that are selected with a view to minimizing bias, thus providing more reliable findings from which conclusions can be drawn and decisions made.

Key characteristics of a systematic review are:

• a clearly defined topic, with pre-defined eligibility criteria for studies
• a systematic and reproducible search strategy
• a critical appraisal of included studies
• data extraction and processing
• analysis and interpretation of results"

Higgins, Julian. Cochrane Handbook for Systematic Reviews of Interventions. Hoboken, NJ, USA: John Wiley & Sons, 2009. https://training.cochrane.org/handbook/archive/v5.0.2/.

1. Define the question

2. Check for recent systematic reviews and protocols

3. Create a review protocol, and register it if appropriate

4. Design and conduct reproducible, comprehensive searches

5. Organize and screen search results

6. Appraise the quality of the studies

7. Extract data

8. Analyze / synthesize data

9. Write the review

Recommended viewing:

• Conducting a systematic review, a short video from Yale's Harvey Cushing/John Hay Whitney Medical Library.
• Introduction to conducting systematic reviews, a 45-minute interactive learning module from Cochrane (requires creation of free login)

Multiple guidelines and standards exist to aid researchers in the creation of high quality systematic reviews and other evidence syntheses.

Reviews may be conducted in accordance to specific internationally-established guidelines, such as Cochrane, the Joanna Briggs Institute, the Campbell Collaboration, and the Collaboration for Environmental Evidence. These follow the most rigorous standards for planning, executing and reporting this type of research. Even if you are not conducting a review for one of these organizations, you may still find it valuable to review their guidance documents. Alternately, reviews may adhere to reporting standards, such as PRISMA or MOOSE. Many of these guidelines have additional guidance on specific review content, such as Cochrane's section on systematic reviews on patient-reported outcomes or JBI's section on Systematic reviews of qualitative evidence.

• Cochrane Collaboration Handbook
  Official document that describes in detail the process of preparing and maintaining Cochrane systematic reviews on the effects of healthcare interventions.

• JBI Manual for Evidence Synthesis
  Official document that describes in detail the process of preparing a JBI systematic or scoping review. Includes instructions for reviews focusing on different types of evidence, such as effectiveness, prevalence, and qualitative evidence. 

• Systematic Reviews: Centre for Reviews and Dissemination’s guidance for undertaking reviews in health care
  Provides practical guidance for undertaking evidence synthesis based on a thorough understanding of systematic review methodology.

• Finding What Works in Health Care: Standards for Systematic Reviews
  Institute of Medicine (IOM)'s standards focus on the development and reporting of comprehensive, publicly funded SRs of the comparative effectiveness of therapeutic medical or surgical interventions.

• Campbell Collaboration
  Systematic reviews of research evidence on the effectiveness of social programs, policies, and practices, including crime and justice, education, international development, and social welfare.

• The PRISMA Statement: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
  Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses. Although it was designed for use with studies of randomized trials, PRISMA can also be used for reporting systematic reviews of other types of research. Please note the PRISMA is a tool for planning and reporting your review, not a guideline or instruction manual. Download the PRISMA Checklist.

• Meta-analysis of Observational Studies in Epidemiology (MOOSE): a proposal for reporting.
  JAMA. 2000 Apr 19;283(15):2008-12.

• AMSTAR checklist
  The AMSTAR checklist can be used to evaluate the methodological quality of an extant systematic review or as a guide when creating a systematic review. Short for "A MeaSurement Tool to Assess systematic Reviews."

• AHRQ Methods Guide for Comparative Effectiveness Research (CER)
  AHRQ definitions and descriptions of methods for systematic reviews of CER
  "For an SR to meet the definition of CER, it should compare at least two alternative interventions, treatments, or systems of care (IOM, 2009b). The interventions and comparators should enable patients and clinicians to balance the benefits and harms of potential treatment options."

• AHRQ Methods Guide for Medical Test Reviews
  AHRQ guide to systematic reviews on the effectiveness of medical tests, with a focus on addressing the challenges of this topic. Training modules at https://effectivehealthcare.ahrq.gov/products/methods-guidance-tests/slides