Systematic reviews and other evidence synthesis projects

• Prepare your data for synthesis
• Determine the heterogeneity of your data (for quantitative data)
• Determine whether changes to your protocol plan are necessary, documenting any made
• Synthesize your data

"Synthesis is a process of bringing together data from a set of included studies with the aim of drawing conclusions about a body of evidence. This will include synthesis of study characteristics and, potentially, statistical synthesis of study findings." - McKenzie JE, Brennan SE, Ryan RE, Thomson HJ, Johnston RV. Chapter 9: Summarizing study characteristics and preparing for synthesis. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.

In this step, you will analyze and synthesize the data extracted from the included studies to address your systematic review question. How you do this depends on the type of studies relevant to your research question. You will have planned this during your protocol and it has guided your data extraction decisions; now you can work on the data itself.

The Cochrane Handbook Chapter 9 lays out a series of steps, paraphrased briefly here:

1. Describe each study including their interventions
2. Determine which are similar enough to be grouped
3. Determine what data from them are available for synthesis, converting their format if necessary
4. Determine whether any modification to the planned comparisons or outcomes is needed, making note of any deviations from your protocol plans
5. Perform synthesis

Reading the full chapter is highly recommended. This page is only a brief outline of the process to help with your planning. The Cochrane chapter goes into extensive detail about how to actually conduct the synthesis. Alternately, you can read the section of the JBI Manual for Evidence Synthesis that corresponds to your review type.

Characteristics of research to take into account as part of an equitable approach for assessment form the acronym PROGRESS(+). These elements are:

• Place of residence
• Race
• Occupation
• Gender
• Religion
• Education
• Social capital
• Socioeconomic position

*The PLUS(+) portion of PROGRESS takes into account other elements of life that serve as ethical considerations when discussing or writing about various health topics and impacts.

For more information about PROGRESS(+), refer to the Equity in Evidence Synthesis tab!

A meta-analysis is only relevant for reviews of study types that produce quantitative data, including mixed-method reviews that use a convergent segregated approach to synthesis and integration.

If this applies to your review, your team will need to have someone experienced with biostatistics. This might be someone who has been involved in all the steps of the review, or someone joining the team just for this step.

To determine whether a meta-analysis is appropriate, it is necessary to first calculate the heterogeneity-- the presence and extent of between-study variation--which affects the extent to which generalizable conclusions can be formed. If heterogeneity is present, your team will need to decide on how to take it into account.

Instructions for performing a meta-analysis are beyond the scope of this guide. Refer instead to Chapter 10 and Chapter 11 in the Cochrane Handbook or the section in the JBI Manual for Evidence Synthesis that corresponds to your review type.

A meta-analysis cannot be performed on qualitative data: a different analysis method must be used. Below are methods for analyzing qualitative data for systematic reviews and other evidence synthesis types.

All Evidence Synthesis Types

• Barnett-Page, E., & Thomas, J. (2009). Methods for the synthesis of qualitative research: a critical review. BMC Med Res Methodol 9(59). https://doi.org/10.1186/1471-2288-9-59
• Cochrane Methods Qualitative and Implementation Group. (n.d.). Core Library of Qualitative Synthesis Methodology. https://methods.cochrane.org/qi/core-library-qualitative-synthesis-methodology
• Thomas, J., & Harden, A. (2008) Methods for the thematic synthesis of qualitative research in systematic reviews. BMC Med Res Methodol 8(45). doi: 10.1186/1471-2288-8-45.

Systematic Reviews

• Johns Hopkins University (n.d.). Introduction to Systematic Reviews and Meta-Analysis-Lecture 6B: What is Qualitative Synthesis - Minimizing Metabias, Qualitative Synthesis, and Interpreting Results. Coursera. https://www.coursera.org/lecture/systematic-review/lecture-6b-what-is-qualitative-synthesis-CHKK7
• Centre for Reviews and Dissemination. (2009). Chapter 6.5: Synthesis of Qualitative Research. In Systematic Reviews - CRD's Guidance for Undertaking reviews in health care.  https://www.york.ac.uk/crd/SysRev/!SSL!/WebHelp/6_5_SYNTHESIS_OF_QUALITATIVE_RESEARCH.htm
• Lockwood C et al. Chapter 2: Systematic reviews of qualitative evidence. (2020). Chapter 2: Systematic reviews of qualitative evidence. In: Aromataris E, Munn Z (Editors). JBI Manual for Evidence Synthesis. https://doi.org/10.46658/JBIMES-20-03
• Soilemezi, D., & Linceviciute, S. (2018).  Synthesizing Qualitative Research: Reflections and Lessons Learnt by Two New Reviewers. International Journal of Qualitative Methods, 17(1). https://doi.org/10.1177/1609406918768014

Rapid Reviews

• Campbell, F., Smith, A., Booth, A., & Weeks, L. (2018). Rapid qualitative reviews (RQRs): what, why, how and where next? Abstracts of the 25th Cochrane Colloquium, Edinburgh, UK. Cochrane Database of Systematic Reviews. https://doi.org/10.1002/14651858.CD201801

Thematic Analysis Tools

• EPPI-Reviewer
• MAXQDA

From University of Toronto Gerstein Science Information Centre