Michelle Gates1, Sarah A Elliott2, Allison Gates3, Meghan Sebastianski4, Jennifer Pillay5, Liza Bialy6, Lisa Hartling7. 1. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-486C Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. mgates1@ualberta.ca. 2. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-488A Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. 3. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-482C Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. 4. Alberta Strategy for Patient-Oriented Research (SPOR) SUPPORT Unit Knowledge Translation Platform, Department of Pediatrics, University of Alberta, 4-486D Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6H 1C9, Canada. 5. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-488D Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. 6. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-486A Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada. 7. Alberta Research Centre for Health Evidence, Department of Pediatrics, University of Alberta, 4-472 Edmonton Clinic Health Academy, 11405-87 Avenue NW, Edmonton, Alberta, T6G 1C9, Canada.
Abstract
BACKGROUND: Living systematic reviews (LSRs) can expedite evidence synthesis by incorporating new evidence in real time. However, the methods needed to identify new studies in a timely manner are not well established. OBJECTIVES: To explore the value of complementary search approaches in terms of search performance, impact on results and conclusions, screening workload, and feasibility compared to the reference standard. METHODS: We developed three complementary search approaches for a systematic review on treatments for bronchiolitis: Automated Full Search, PubMed Similar Articles, and Scopus Citing References. These were automated to retrieve results monthly; pairs of reviewers screened the records and commented on feasibility. After 1 year, we conducted a full update search (reference standard). For each complementary approach, we compared search performance (proportion missed, number needed to read [NNR]) and reviewer workload (number of records screened, time required) to the reference standard. We investigated the impact of the new trials on the effect estimate and certainty of evidence for the primary outcomes. We summarized comments about feasibility. RESULTS: Via the reference standard, reviewers screened 505 titles/abstracts, 24 full texts, and identified four new trials (NNR 127; 12.4 h). Of the complementary approaches, only the Automated Full Search located all four trials; these were located 6 to 12 months sooner than via the reference standard but did not alter the results nor certainty in the evidence. The Automated Full Search was the most resource-intensive approach (816 records screened; NNR 204; 17.1 h). The PubMed Similar Articles and Scopus Citing References approaches located far fewer records (452 and 244, respectively), thereby requiring less screening time (9.4 and 5.2 h); however, each approach located only one of the four new trials. Reviewers found it feasible and convenient to conduct monthly screening for searches of this yield (median 15-65 records/month). CONCLUSIONS: The Automated Full Search was the most resource-intensive approach, but also the only to locate all of the newly published trials. Although the monthly screening time for the PubMed Similar Articles and Scopus Citing Articles was far less, most relevant records were missed. These approaches were feasible to integrate into reviewer work processes. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework. https://doi.org/10.17605/OSF.IO/6M28H .
BACKGROUND: Living systematic reviews (LSRs) can expedite evidence synthesis by incorporating new evidence in real time. However, the methods needed to identify new studies in a timely manner are not well established. OBJECTIVES: To explore the value of complementary search approaches in terms of search performance, impact on results and conclusions, screening workload, and feasibility compared to the reference standard. METHODS: We developed three complementary search approaches for a systematic review on treatments for bronchiolitis: Automated Full Search, PubMed Similar Articles, and Scopus Citing References. These were automated to retrieve results monthly; pairs of reviewers screened the records and commented on feasibility. After 1 year, we conducted a full update search (reference standard). For each complementary approach, we compared search performance (proportion missed, number needed to read [NNR]) and reviewer workload (number of records screened, time required) to the reference standard. We investigated the impact of the new trials on the effect estimate and certainty of evidence for the primary outcomes. We summarized comments about feasibility. RESULTS: Via the reference standard, reviewers screened 505 titles/abstracts, 24 full texts, and identified four new trials (NNR 127; 12.4 h). Of the complementary approaches, only the Automated Full Search located all four trials; these were located 6 to 12 months sooner than via the reference standard but did not alter the results nor certainty in the evidence. The Automated Full Search was the most resource-intensive approach (816 records screened; NNR 204; 17.1 h). The PubMed Similar Articles and Scopus Citing References approaches located far fewer records (452 and 244, respectively), thereby requiring less screening time (9.4 and 5.2 h); however, each approach located only one of the four new trials. Reviewers found it feasible and convenient to conduct monthly screening for searches of this yield (median 15-65 records/month). CONCLUSIONS: The Automated Full Search was the most resource-intensive approach, but also the only to locate all of the newly published trials. Although the monthly screening time for the PubMed Similar Articles and Scopus Citing Articles was far less, most relevant records were missed. These approaches were feasible to integrate into reviewer work processes. SYSTEMATIC REVIEW REGISTRATION: Open Science Framework. https://doi.org/10.17605/OSF.IO/6M28H .
Entities:
Keywords:
Evidence synthesis; Evidence-based medicine; Knowledge synthesis; Literature searching; Living systematic review; Methods; Systematic review; Updating
Authors: Kaveh G Shojania; Margaret Sampson; Mohammed T Ansari; Jun Ji; Steve Doucette; David Moher Journal: Ann Intern Med Date: 2007-07-16 Impact factor: 25.391
Authors: Peter Bragge; Ornella Clavisi; Tari Turner; Emma Tavender; Alex Collie; Russell L Gruen Journal: BMC Med Res Methodol Date: 2011-06-17 Impact factor: 4.615
Authors: Julian H Elliott; Anneliese Synnot; Tari Turner; Mark Simmonds; Elie A Akl; Steve McDonald; Georgia Salanti; Joerg Meerpohl; Harriet MacLehose; John Hilton; David Tovey; Ian Shemilt; James Thomas Journal: J Clin Epidemiol Date: 2017-09-11 Impact factor: 6.437