Semi-Automating Systematic Review Queries

by Valerie Lookingbill

With the exponential rise in online publishing, there is an increasing need for comprehensive overviews of research, including systematic reviews, a methodology for conducting reviews of literature that prescribes explicit, reproducible, and transparent processes for collating the best available evidence on a particular topic. While this rise in publications presents opportunities for researchers conducting systematic reviews, it also presents unique challenges. The systematic review process entails several explicit steps, including the use of robust techniques for searching for and identifying studies. Traditionally, the pipeline of systematic reviews begins with information professionals manually identifying key concepts and conducting comprehensive searches in multiple databases using keywords and controlled vocabulary to retrieve potentially relevant references. However, developing search strategies for systematic reviews is an intensive, time-consuming process involving significant human effort. Previous research (van de Schoot et al., 2021) has suggested, though, that the “future of systematic reviewing will be an interaction with machine learning algorithms” (p. 131), in which machine learning can be applied to the systematic review process to decrease manual labor and ultimately increase efficiency. As such, the proposed project, outlined during the 2022 IDEA Institute on Artificial Intelligence, aims to reduce the human workload in developing search strategies for systematic reviews by using machine learning methods.

There have been, and continue to be, efforts to apply machine learning to systematic reviews, mainly through the development of software tools. These efforts have predominantly applied machine learning methods in the screening stages of the reviews (Lange et al., 2021; van de Schoot et al., 2021), as well as data extraction and risk of bias assessment (Blaizot et al., 2022). However, there still exists a need to apply machine learning methods to aid information professionals in constructing the complex queries required of systematic reviews, particularly as the retrieval stage of the review process commonly results in high-recall and low-precision (Schells et al., 2019). The proposed project would, thus, address this gap. Through the development of a semi-automated software tool, a user could input a research question and examples articles to be retrieved in a search. The user could then use the tool to develop an initial Boolean query and, after the query is finalized, translate that query across multiple databases using a combination of keywords and resource-specific controlled vocabulary.

The IDEA Institute on Artificial Intelligence has been an incredible experience as it has afforded me the conceptual knowledge and the confidence to go back to my institution and begin having conversations about applying machine learning to the work that we already do. As with other libraries, machine learning methods can help us improve our research services and enable us to serve our patrons better.

References

Blaizot, A., Veettil, S. K., Saidoung, P., Moreno‐Garcia, C. F., Wiratunga, N., Aceves‐Martins, M., Lai, N. M., & Chaiyakunapruk, N. (2022). Using artificial intelligence methods for systematic review in health sciences: A systematic review. Research Synthesis Methods, 13(3), 353-362.

Lange, T., Schwarzer, G., Datzmann, T., & Binder, H. (2021). Machine learning for identifying relevant publications in updates of systematic reviews of diagnostic test studies. Research Synthesis Methods, 12(4), 506-515.

van de Schoot, R., de Bruin, J., Schram, R., Zahedi, P., de Boer, J., Weijdema, F., Kramer, B., Huijts, M., Hoogerwerf, M., Ferdinands, G., Harkema, A., Willemsen, J., Ma, Y., Fang, O., Hindriks, S., Tummers, L., & Oberski, D. L. (2021). An open source machine learning framework for efficient and transparent systematic reviews. Nature Machine Intelligence, 3(2), 125-133.