Systematic reviews and other evidence synthesis projects
Introduction to Artificial Intelligence
How do we define Artificial Intelligence (AI)?
It is the theory and development of computer system able to perform tasks that normally require human intelligence, such as visual perception, speech recognition, decision-making, and translation between languages.
Examples of AI:
- Automated synthesis of information
- Natural language processing
- Speech and text analysis
- Facial recognition
Purpose and Strategies
Purpose
- Using AI as a mediating step in between sections of the systematic review process
- Creates efficient operations and reduces the amount of time spent on more time-heavy portions
- Using AI as an aid to make faster decisions
- Increasing transparency and clarity in review questions
Strategies
- Determine the strengths and weaknesses of different sections of the systematic review process
- Identify the areas that take the most amount of time
- Assess the risk in automation
- Talk to research and library team about where automated processes would benefit in the process
AI in Systematic Review Process
AI in Systematic Review Process
Human Review Primary (in between first and second step):
- AI can synthesize information to form a protocol
- Checking to make sure elements of DEI are included in protocol and all components are presents
Human Review Secondary (in between second and third step):
- Autogenerated search strings
- Automated literature selections; Conducting the quality check after return results
Human Review Tertiary (in between third and fourth step):
- Automated selection of studies; review selection criteria and process
- Automated data extraction; review type of data and what is included and excluded
- Automated synthesis of data; review for any biases and exclusions
Benefits and Challenges
Machine Bias
- Overestimations of research data input
- Inaccurate or unfair predictions
- Information exclusion
- Overspecification
- Discrimination against specific groups
Research Bias
- Lack of representation for marginalized groups in medical research
- Grey literature may not always be considered
Equity Considerations
- May not consider equitable practices
- With the presence of machine discrimination, equity may go out the window
- Equity can be highlighted from the human lens
In an effort to strengthen the processes that use AI, it is important to provide feedback and speak up about any inconsistencies or biases noticed in the intermediate reviews. Also, always remember to assess the role of AI in your project and document when it was used in your methods section.
Resources
- An open source machine learning framework for efficient and transparent systematic reviews
- Using artificial intelligence methods for systematic review in health sciences: A systematic review
- The Case for Artificial Intelligence in Systematic Reviews
- Systematic Review Writing by Artificial Intelligence: Can Artificial Intelligence Replace Humans?
- Rayyan AI Systematic Review System (a company that conducts systematic reviews with AI processing)
- Artificial intelligence and automation of systematic reviews in women's health
- Can Systematic Reviews Be Automated?
- Artificial Intelligence in Systematic Reviews: Evaluation of Emerging Electronic Tools for Researchers in Knowledge Synthesis
- Abstrackr: Software for Semi-Automatic Citation Screening
- OpenMetaAnalyst: Powerful Open-Source Software for Meta-Analysis
- Text mining for searching and screening the literature: https://libraryguides.mcgill.ca/text-mining/home
- Clark J, McFarlane C, Cleo G, Ishikawa Ramos C, Marshall S. The Impact of Systematic Review Automation Tools on Methodological Quality and Time Taken to Complete Systematic Review Tasks: Case Study. JMIR Med Educ. 2021 May 31;7(2):e24418. doi: 10.2196/24418. PMID: 34057072; PMCID: PMC8204237.
- Cierco Jimenez R, Lee T, Rosillo N, Cordova R, Cree IA, Gonzalez A, Indave Ruiz BI. Machine learning computational tools to assist the performance of systematic reviews: A mapping review. BMC Med Res Methodol. 2022 Dec 16;22(1):322. doi: 10.1186/s12874-022-01805-4. PMID: 36522637; PMCID: PMC9756658. https://pubmed.ncbi.nlm.nih.gov/36522637/
Guide Design Credit
Dev Wilder UW MLIS Candidate 2023