Ivan Lerner1, Perrine Créquit2, Philippe Ravaud3, Ignacio Atal4. 1. Centre de Recherche Epidémiologie et Statistique Paris Sorbonne Cité, INSERM U1153, Paris, France; Université Paris Descartes - Sorbonne Paris cité, Paris, France; Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Centre d'Epidémiologie Clinique, Paris, France. 2. Centre de Recherche Epidémiologie et Statistique Paris Sorbonne Cité, INSERM U1153, Paris, France; Université Paris Descartes - Sorbonne Paris cité, Paris, France; Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Centre d'Epidémiologie Clinique, Paris, France; Cochrane France, Paris, France. 3. Centre de Recherche Epidémiologie et Statistique Paris Sorbonne Cité, INSERM U1153, Paris, France; Université Paris Descartes - Sorbonne Paris cité, Paris, France; Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Centre d'Epidémiologie Clinique, Paris, France; Cochrane France, Paris, France; Department of Epidemiology, Mailman School of Public Health, Columbia University New York, USA. 4. Centre de Recherche Epidémiologie et Statistique Paris Sorbonne Cité, INSERM U1153, Paris, France; Université Paris Descartes - Sorbonne Paris cité, Paris, France; Hôpital Hôtel-Dieu, Assistance Publique-Hôpitaux de Paris, Centre d'Epidémiologie Clinique, Paris, France. Electronic address: ignacio.atal-ext@aphp.fr.
Abstract
OBJECTIVES: We aimed to develop and evaluate an algorithm for automatically screening citations when updating living network meta-analysis (NMA). STUDY DESIGN AND SETTING: Our algorithm learns from the initial screening of citations conducted when creating an NMA to automatically identify eligible citations (i.e., needing full-text consideration) when updating the NMA. We evaluated our algorithm on four NMAs from different medical domains. For each NMA we constructed sets of initially screened citations and citations to screen during an update that took place 2 years after the conduct of the NMA. We encoded free text of citations (title and abstract) using word embeddings. On top of this vectorized representation, we fitted a logistic regression model to the set of initially screened citations to predict the eligibility of citations screened during an update. RESULTS: Our algorithm achieved 100% sensitivity on two NMAs (100% [95% confidence interval 93-100] and 100% [40-100] sensitivity), and 94% (81-99) and 97% (86-100) on the remaining two others. For all NMAs, our algorithm would have spared to manually screen 1,345 of 2,530 citations, decreasing the workload by 53% (51-55), while missing 3 of 124 eligible citations (2% [1-7]), none of which were finally included in the NMAs after full-text consideration. CONCLUSION: For updating an NMA after 2 years, our algorithm considerably diminished the workload required for screening, and the number of missed eligible citations remained low.
OBJECTIVES: We aimed to develop and evaluate an algorithm for automatically screening citations when updating living network meta-analysis (NMA). STUDY DESIGN AND SETTING: Our algorithm learns from the initial screening of citations conducted when creating an NMA to automatically identify eligible citations (i.e., needing full-text consideration) when updating the NMA. We evaluated our algorithm on four NMAs from different medical domains. For each NMA we constructed sets of initially screened citations and citations to screen during an update that took place 2 years after the conduct of the NMA. We encoded free text of citations (title and abstract) using word embeddings. On top of this vectorized representation, we fitted a logistic regression model to the set of initially screened citations to predict the eligibility of citations screened during an update. RESULTS: Our algorithm achieved 100% sensitivity on two NMAs (100% [95% confidence interval 93-100] and 100% [40-100] sensitivity), and 94% (81-99) and 97% (86-100) on the remaining two others. For all NMAs, our algorithm would have spared to manually screen 1,345 of 2,530 citations, decreasing the workload by 53% (51-55), while missing 3 of 124 eligible citations (2% [1-7]), none of which were finally included in the NMAs after full-text consideration. CONCLUSION: For updating an NMA after 2 years, our algorithm considerably diminished the workload required for screening, and the number of missed eligible citations remained low.
Authors: Zhengyi Deng; Kanhua Yin; Yujia Bao; Victor Diego Armengol; Cathy Wang; Ankur Tiwari; Regina Barzilay; Giovanni Parmigiani; Danielle Braun; Kevin S Hughes Journal: JCO Clin Cancer Inform Date: 2019-08
Authors: Christopher R Norman; Elizabeth Gargon; Mariska M G Leeflang; Aurélie Névéol; Paula R Williamson Journal: Database (Oxford) Date: 2019-01-01 Impact factor: 3.451