Skip to Main Content

Systematic reviews and other evidence synthesis projects

To Do during this step

  • Combine and de-duplicate your search results
  • Title/abstract screening by 2 or more people
  • Full-text screening by 2 or more people
  • Conduct citation searching on included articles
  • Record the data from these steps in PRISMA flow diagram

Pro tip: Save time and stress by piloting the title/abstract and full-text screening with the whole team. There will be fewer conflicts to resolve later.

Reporting Your Process

You will need to report your screening process. Use these tools to record how many results are removed during each stage of screening.

The PRISMA Statement: Preferred Reporting Items for Systematic Reviews and Meta-Analyses
Evidence-based minimum set of items for reporting in systematic reviews and meta-analyses.  Focused on randomized trials,  PRISMA can also be used as a basis for reporting systematic reviews of other types of research, particularly evaluations of interventions.

The PRISMA flow diagram is a tool for recording and reporting the number of records during the different steps of a systematic review, along with reasons for exclusion. It is often included within the review or as supplemental material.

Tool for Generating a PRISMA Search Flow Diagram - from the Evidence Synthesis Hackathon

Complying with PRISMA and Standards for Cochrane: See Cochrane Handbook > Methodological Expectations of Cochrane Intervention Reviews (MECIR)

Managing References

Manage Your Search Results

Export the results of your search from each database. For instructions, see the guide for your database in the Database Searching guides or check the Help or Support for the database.

Import the file into your citation management tool. For instructions, see Citation Management Tools for guides on using EndNote, EndNote Web and other citation management programs for importing, and for storing and organizing search results. EndNote Desktop and Zotero can interface with the UW Libraries catalog to automatically download many of the pdfs for your full text screening.


De-duplicating

In your searches across multiple databases, there will be some articles that are retrieved in more than one database. This is expected, but you don't need to evaluate that article more than once. Before you start screening, de-duplicate. The processes described below are methods for removing duplicates while minimizing the risk of accidentally removing similar non-duplicate articles. Some of the screening tools also have effective de-duplication.

SRA-DM tool:  Rathbone J, Carter M, Hoffmann T, Glasziou P.  Better duplicate detection for systematic reviewers: evaluation of Systematic Review Assistant-Deduplication Module. Syst Rev. 2015 Jan 14;4:6. doi: 10.1186/2046-4053-4-6. PubMed PMID: 25588387.

For desktop version of EndNote only: Bramer WM, Giustini D, de Jonge GB, Holland L, Bekhuis T. De-duplication of database search results for systematic reviews in EndNote. Journal of the Medical Library Association : JMLA. 2016;104(3):240-243. doi:10.3163/1536-5050.104.3.014<

Make a note of how many duplicates were removed for reporting in your paper. Your PRISMA flow diagram is a good place to keep track.

Screening the Articles

Steps for screening

The purpose of article screening to remove studies that are not eligible for inclusion.

Use your inclusion/exclusion criteria, two or more team members will conduct the following:

  1. Title/abstract screening: First, screen the title and abstracts of the studies and determine whether they are relevant to your research question. Since you conducted a comprehensive search, there will be items that were captured that are clearly not relevant.
  2. Full text screening: For studies included based on the title/abstract screening, obtain the full text and evaluate for inclusion/exclusion.

During both steps, record the reason for excluding an item. Review support software commonly contains features to simplify this.

Review support software will typically include a record screening/study selection function. This allows more than one reviewer to independently screen the records without seeing other reviewers' decisions to include or exclude studies, and thus reduces bias. Areas of disagreement can be resolved by consensus or by a third party who is an expert in the field.

Project management software for systematic reviews and other evidence synthesis:

These tools provide support for independent screening of the title/abstracts and the full text of articles. Some have additional features, such as support for data extraction or machine-learning to sort results. Additional tools are described in the Wu et al. poster below.

Chart showing the features of different tools and which step during which they are relevant:
"Digital Tools for Managing Different Steps of the Systematic Review Process". Wu W, Akers K, Hu E, Sarkozy A, Vinson P. Library Scholarly Publications. 2018; 136. https://digitalcommons.wayne.edu/libsp/136

Reviewing retrieved references for inclusion in systematic reviews using Endnote. Bramer WM, Milic J, Mast F. J Med Libr Assoc. 2017 Jan; 105(1): 84-87.

Reference checking for systematic reviews using Endnote. Bramer WM. J Med Libr Assoc. 2018 Oct; 106(4): 542-6.

Covidence and Rayyan. Kellermeyer L, Harnke B, Knight S. J Med Libr Assoc. 2018 Oct; 106(4): 580-3. (Review)

U Conn's guide to using Rayyan

Covidence training resources

Translation Services

The UW does not have an official translation service for research articles. The following resources are worth trying, but may not be able to provide everything you need.

Citation Searching

Once you have completed the full-text screening, you will use the included articles to identify additional potentially similar articles in a process often called citation searching or citation chasing. This is done on the basis that it is probable that studies which cite or are cited by a source study will contain similar content. This practice is recommended in section 1.1.4 of the Technical Supplement to Chapter 4 of the 2022 Cochrane Handbook.

- Backward citation searching: Consult the reference lists for the included articles. Locate the title and abstract information for the references, then screen them according to your screening criteria.

- Forward citation searching: Locate articles which cite your included articles, then screen them according to your screening criteria.

Tools to make this easier:

  • SpiderCite from SR-Accelerator
    Export your included articles as an EndNote library, then import them into SpiderCite. SpiderCite will generate libraries of the references and the articles citing your included articles that can be imported back into EndNote for review.
  • citationchaser by Neal Haddaway
    Paste in a list of the doi's or PMIDs for your included articles or import as an Excel or RIS file. citationchaser will generate RIS libraries of the references and the articles citing your included articles that can be imported into a citation manager or screening software for review.
  • OpenAlex from OurResearch (in development)
    Mentioned in the 2022 Cochrane Handbook. Its API is currently operative, but as of Sept. 2022 its website is still in progress.
  • Reference checking for systematic reviews using Endnote. Bramer WM. J Med Libr Assoc. 2018 Oct; 106(4): 542-6.

Databases that are especially good for forward citation searching:

  • Web of Science Core Collection
    Web of Science will only identify citing articles that are also in Web of Science, but since it has over 80 million records it is still thorough. You can search on individual titles or paste in a string of the doi's or PMIDs for your included articles joined with OR. The entry for each will let you get a list of the articles that cited that article which you can export. It will also identify articles citing any conference proceedings you have in your included materials.
  • Scopus
    The UW does not have access to Scopus, but if you are on a SR team with other institutions, it may be an option. Features and strategies are similar to those for Web of Science.

 

Lefebvre C, Glanville J, Briscoe S, Featherstone R, Littlewood A, Marshall C, Metzendorf M-I, Noel-Storr A, Paynter R, Rader T, Thomas J, Wieland LS. Technical Supplement to Chapter 4: Searching for and selecting studies. In: Higgins JPT, Thomas J, Chandler J, Cumpston M, Li T, Page MJ, Welch VA (editors). Cochrane Handbook for Systematic Reviews of Interventions version 6.3 (updated February 2022). Cochrane, 2022. Available from www.training.cochrane.org/handbook.