Sandra McKeown1, Zuhaib M Mir2. 1. Bracken Health Sciences Library, Queen's University, 18 Stuart Street, Kingston, Ontario, K7L 2V5, Canada. sandra.mckeown@queensu.ca. 2. Departments of Surgery and Public Health Sciences, Queen's University & Kingston Health Sciences Centre, 76 Stuart Street, Kingston, Ontario, K7L 2V7, Canada.
Abstract
BACKGROUND: Systematic reviews involve searching multiple bibliographic databases to identify eligible studies. As this type of evidence synthesis is increasingly pursued, the use of various electronic platforms can help researchers improve the efficiency and quality of their research. We examined the accuracy and efficiency of commonly used electronic methods for flagging and removing duplicate references during this process. METHODS: A heterogeneous sample of references was obtained by conducting a similar topical search in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and PsycINFO databases. References were de-duplicated via manual abstraction to create a benchmark set. The default settings were then used in Ovid multifile search, EndNote desktop, Mendeley, Zotero, Covidence, and Rayyan to de-duplicate the sample of references independently. Using the benchmark set as reference, the number of false-negative and false-positive duplicate references for each method was identified, and accuracy, sensitivity, and specificity were determined. RESULTS: We found that the most accurate methods for identifying duplicate references were Ovid, Covidence, and Rayyan. Ovid and Covidence possessed the highest specificity for identifying duplicate references, while Rayyan demonstrated the highest sensitivity. CONCLUSION: This study reveals the strengths and weaknesses of commonly used de-duplication methods and provides strategies for improving their performance to avoid unintentionally removing eligible studies and introducing bias into systematic reviews. Along with availability, ease-of-use, functionality, and capability, these findings are important to consider when researchers are selecting database platforms and supporting software programs for conducting systematic reviews.
BACKGROUND: Systematic reviews involve searching multiple bibliographic databases to identify eligible studies. As this type of evidence synthesis is increasingly pursued, the use of various electronic platforms can help researchers improve the efficiency and quality of their research. We examined the accuracy and efficiency of commonly used electronic methods for flagging and removing duplicate references during this process. METHODS: A heterogeneous sample of references was obtained by conducting a similar topical search in MEDLINE, Embase, Cochrane Central Register of Controlled Trials, and PsycINFO databases. References were de-duplicated via manual abstraction to create a benchmark set. The default settings were then used in Ovid multifile search, EndNote desktop, Mendeley, Zotero, Covidence, and Rayyan to de-duplicate the sample of references independently. Using the benchmark set as reference, the number of false-negative and false-positive duplicate references for each method was identified, and accuracy, sensitivity, and specificity were determined. RESULTS: We found that the most accurate methods for identifying duplicate references were Ovid, Covidence, and Rayyan. Ovid and Covidence possessed the highest specificity for identifying duplicate references, while Rayyan demonstrated the highest sensitivity. CONCLUSION: This study reveals the strengths and weaknesses of commonly used de-duplication methods and provides strategies for improving their performance to avoid unintentionally removing eligible studies and introducing bias into systematic reviews. Along with availability, ease-of-use, functionality, and capability, these findings are important to consider when researchers are selecting database platforms and supporting software programs for conducting systematic reviews.
Authors: Margaret Sampson; Nicholas J Barrowman; David Moher; Terry P Klassen; Ba' Pham; Robert Platt; Philip D St John; Raymond Viola; Parminder Raina Journal: J Clin Epidemiol Date: 2003-10 Impact factor: 6.437
Authors: Christopher W Halladay; Thomas A Trikalinos; Ian T Schmid; Christopher H Schmid; Issa J Dahabreh Journal: J Clin Epidemiol Date: 2015-02-07 Impact factor: 6.437
Authors: Filomena Gomes; Gilles Bergeron; Megan W Bourassa; Diana Dallmann; Jenna Golan; Kristen M Hurley; Shannon E King; Ana Carolina Feldenheimer da Silva; Saurabh Mehta Journal: Ann N Y Acad Sci Date: 2020-02-13 Impact factor: 5.691
Authors: Melanie Lum; Luke Wolfenden; Jannah Jones; Alice Grady; Hayley Christian; Kathryn Reilly; Sze Lin Yoong Journal: Int J Environ Res Public Health Date: 2022-02-10 Impact factor: 3.390
Authors: Shamsudeen Mohammed; Laura L Oakley; Milly Marston; Judith R Glynn; Clara Calvert Journal: J Glob Health Date: 2022-09-03 Impact factor: 7.664