Systematic reviews and other evidence synthesis projects

• Develop extraction form
• Train data extractors to use the data collection form or system
• Data extraction by 2 or more people

Pro tip: Save time and stress by piloting your data extraction form with the whole team. If there are any fields missing or unclear, it's much easier to fix them now.

In your protocol

Your protocol includes a description of how you will extract data from your screened and appraised articles.

Some elements you will need to decide about data extraction plan include:

• Who - independent duplicate extraction by 2 or more reviewers is recommended, as with the screening step
• What data will be extracted - descriptions and definitions
• Any standardized forms or checklists to be used
• Technology - some review support software has a data extraction feature, but you can also use Excel or even pencil & paper
• Method of obtaining any missing data for included articles

For reporting your data extraction plan in your protocol, see items PR23-PR26 in Cochrane's MECIR Manual or items 11c-13 in the PRISMA-P Explanation and Elaboration paper.

In your review

For a full description of the data extraction step, see Chapter 5 in the Cochrane Handbook for Systematic Reviews of Interventions version 6.3, or one of the other resources below:

• Chapter 5: Collecting Data (Cochrane Handbook)
• Data Extraction Training Module PowerPoint (AHRQ)
• Undertaking the Review (University of York); see section 1.3.5 Data Synthesis

What should I extract?

You should plan to extract only data that is relevant to answering the question posed in your systematic review. It may help to consult other similar systematic reviews to identify what data to collect.  You should use your key questions and your eligibility criteria as a starting point.  It can also help to think about your question in a framework such as PICO.

Data to be extracted may include:

• Information about the study (author(s), year of publication, title, DOI)
• Demographics (age, sex, ethnicity, diseases / conditions, other characteristics related to the intervention / outcome)
• Methodology (study type, participant recruitment / selection / allocation, level of evidence, study quality)
• Intervention (quantity, dosage, route of administration, format, duration, time frame, setting)
• Outcomes (quantitative and / or qualitative)

If you plan to synthesize data, you will want to collect additional information such as sample sizes, effect sizes, dependent variables, reliability measures, pre-test data, post-test data, follow-up data, and statistical tests used.

Equitable Considerations

This is the step where you go back through your list of intervention intentions and match them with the data to extract. Having these present will guide you in gathering data as well as noticing any gaps in the research you are assessing. Here are some questions you can think about and address at this stage of the process:

• Was there something you hoped to see in the paper that was not included?

This is the time to make note of those things and continue with data extraction!

• What is the point of reporting on information that is not in data at the point of extraction?

The point is to provide transparency and a full context. This can highlight underlying issues indirectly related to the topic.

• What is the overarching goal of this?

The goal is to talk about your procedure and your reasons for research while also acknowledging some of the information that may need more information. 

Data extraction tips

1. Examine an article that definitely belongs in the review: what data elements would you want to capture from it?
2. Look for an existing extraction form or tool to help guide you. Use existing systematic reviews on your topic to identify what information to collect if you are not sure what to do.
3. Train the review team on the extraction categories and what type of data would be expected. A manual or guide may help your team establish standards.
4. Pilot the extraction / coding form to ensure data extractors are recording similar data. Revise the extraction form if needed.
5. Discuss any discrepancies in coding throughout the process.
6. Document any changes to the process or the form. Keep track of the decisions the team makes and the reasoning behind them.

Tools

The following review support software includes data extraction features:

Covidence
Covidence is web-based software including data extraction and consensus. Easy export to popular statistical analysis programs.

• Download "A practical guide to Data Extraction for Intervention Systematic Reviews" (emphasis on extracting data on intervention studies but some of the info can be generalized to all systematic reviews)
• Read more about its broadly customizable extraction tool, Extraction 2.0.
• Learn more about UW's subscription to Covidence

DistillerSR
DistillerSR is web-based software with support for data extraction and analysis. Algorithms assist during screening and extraction. It guides reviewers in creating project-specific forms, extracting, and analyzing data. Free trial for individual students; fee-based for faculty and longer student projects.

Colandr
Colandr is free web-based software with support for data extraction and export in CSV format (openable in Excel). Algorithms assist during screening and extraction.

RevMan
RevMan is free software with support for data extraction and analysis. It offers collection forms for descriptive information on population, interventions, and outcomes, and quality assessments, as well as for data for analysis and forest plots. Has a web-based and a desktop version.

Sumari
JBI Sumari is web-based software with support for data extraction and synthesis. Fee-based; free 14-day trial.

The Systematic Review Toolbox:
The SR Toolbox  is a community-driven, searchable, web-based catalogue of tools that support the systematic review process across multiple domains. Use the advanced search option to restrict to tools specific to data extraction.

Just extraction

SRDR+
SRDR+  is free web-based software for the extraction and management of data for systematic review or meta-analysis. It is also an open and searchable archive of systematic reviews and their data, and has a training environment, tutorials, and example templates of systematic review data extraction forms. Access the "Extracting Data" tutorial for more information.

Excel
You can create custom extraction forms in Excel. Spreadsheet functions such as drop-down menus and range checks can speed up the process and help prevent data entry errors. Relational databases (such as Microsoft Access) can help you extract information from different categories like citation details, demographics, participant selection, intervention, outcomes, etc. A more advanced approach to using Excel for this purpose is the PIECES approach, designed by a librarian at Texas A&M. The PIECES workbook is downloadable at this guide.

To record the data, you can also use online survey forms such as RedCAP, Qualtrics, or Survey Monkey, or design and create your own coded fillable forms using Adobe Acrobat Pro or Microsoft Access.

Templates

• Data Extraction Template (Cochrane Working Group)
• Data collection form for intervention reviews for RCTs and non-RCTs - template (Cochrane Working Group)
• JBI Manual for Evidence Synthesis - Several of the systematic review types have corresponding evidence forms

Guidance for creating your own

• Brown SA, Upchurch SL, Acton GJ. A framework for developing a coding scheme for meta-analysis. West J Nurs Res. 2003;25(2):205-222. doi:10.1177/0193945902250038: "The purpose of this article is to describe the data coding instruments and strategies used to establish reliability and validity of these instruments. Initial instrument development is described, followed by a discussion of how the instruments were adapted for subsequent studies."
• Brown SA, Martin EE, Garcia TJ, et al. Managing complex research datasets using electronic tools: a meta-analysis exemplar. Comput Inform Nurs. 2013;31(6):257-265. doi:10.1097/NXN.0b013e318295e69c.

What should you use?

Elamin MB, Flynn DN, Bassler D, et al. Choice of data extraction tools for systematic reviews depends on resources and review complexity. J Clin Epidemiol. 2009;62(5):506-510. doi:10.1016/j.jclinepi.2008.10.016. - Advice on how to decide what electronic tools to use to extract data for analytical reviews

Modified from UNC Health Sciences Library and Cornell University Library