Jason W Busse1, Paul Bruno2, Keshena Malik3, Gaelan Connell4, David Torrance4, Trung Ngo4, Karin Kirmayr5, Daniel Avrahami4, John J Riva6, Shanil Ebrahim7, Peter A A Struijs8, David Brunarski9, Stephen J Burnie4, Frances LeBlanc10, Eric A Coomes11, Ivan A Steenstra12, Tesha Slack13, Robert Rodine14, Janey Jim15, Victor M Montori16, Gordon H Guyatt17. 1. Department of Anesthesia, McMaster University, HSC-2U1, 1200 Main Street West, Hamilton, Ontario, Canada L8S 4K1; Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1; The Michael G. DeGroote Institute for Pain Research and Care, McMaster University, MDCL 2101, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1. Electronic address: bussejw@mcmaster.ca. 2. Faculty of Kinesiology and Health Studies, University of Regina, 3737 Wascana Pkwy, Regina, Saskatchewan, Canada S4S 0A2. 3. School of Rehabilitation Science, McMaster University, 1400 Main St. W., Hamilton, Ontario, Canada L8S 1C7. 4. Canadian Memorial Chiropractic College, 6100 Leslie St, Toronto, Ontario, Canada M2H 3J1. 5. German Hospital, 1640 Pueyrredón, C1118 AAT, Buenos Aires, Argentina. 6. Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1; Department of Family Medicine, McMaster University, The Atrium at McMaster Innovation Park, 175 Longwood Road South, Hamilton, Ontario, Canada L8P 0A1. 7. Department of Anesthesia, McMaster University, HSC-2U1, 1200 Main Street West, Hamilton, Ontario, Canada L8S 4K1; Department of Clinical Epidemiology and Biostatistics, McMaster University, 1280 Main Street West, Hamilton, Ontario, Canada L8S 4K1; Stanford Prevention Research Center, Department of Medicine, Stanford University, 1265 Welch Road, Stanford, California, USA 94305; Department of Anaesthesia & Pain Medicine, The Hospital for Sick Children, 555 University Ave, Toronto, Ontario, Canada M5G 1X8. 8. Department of orthopaedic surgery, Academic Medical Center, Meibergdreef 9, 1105 AZ, Amsterdam, The Netherlands. 9. Ontario Chiropractic Association, 20 Victoria St., Suite 200 Toronto, Ontario, Canada M5C 2N8. 10. Canadian Memorial Chiropractic College, 6100 Leslie St, Toronto, Ontario, Canada M2H 3J1; Canadian Chiropractic Association, 186 Spadina Avenue, Suite 6, Toronto, Ontario, Canada M5T 3B2. 11. University of Toronto, Faculty of Medicine, 1 King's College Circle Toronto, Ontario, Canada M5S 1A8. 12. The Institute for Work & Health, 8th floor, 481 University Ave., Toronto, Ontario, Canada M5G 2E9; Dalla Lana School of Public Health, University of Toronto, 6th floor, 155 College St, Toronto, Ontario, Canada M5T 3M7. 13. The Institute for Work & Health, 8th floor, 481 University Ave., Toronto, Ontario, Canada M5G 2E9. 14. Restorative Health, 12 William St W, Smiths Falls, Ontario, Canada K7A 1M9. 15. Jointworks Chiropractic Inc., 3860 Main St, Vancouver, British Columbia, Canada V5V 3N9. 16. Knowledge and Evaluation Research Unit, Mayo Clinic, 200 First Street SW, Plummer 3-35, Rochester, MN, USA 55905. 17. Department of Anesthesia, McMaster University, HSC-2U1, 1200 Main Street West, Hamilton, Ontario, Canada L8S 4K1.
Abstract
OBJECTIVE: To assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review. STUDY DESIGN AND SETTINGS: Systematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non-English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility. RESULTS: Of 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity=0.89; specificity=0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word "random" appears) led to accurate classification of 51 of 53 articles (sensitivity=0.96; specificity=0.70). CONCLUSION: Our findings show promise for limiting the need for non-English-language review teams in systematic reviews with large numbers of potentially eligible non-English-language articles.
OBJECTIVE: To assess English-speaking reviewers' accuracy in determining the eligibility of foreign-language articles for a systematic review. STUDY DESIGN AND SETTINGS: Systematic review of randomized controlled trials of therapy for fibromyalgia. Guided by 10 questions, English-speaking reviewers screened non-English-language articles for eligibility. Teams of two native-language speakers provided reference standard judgments of eligibility. RESULTS: Of 15,466 potentially eligible articles, we retrieved 763 in full text, of which 133 were published in 19 non-English languages; 53 trials published in 11 languages other than English proved eligible. Of the 53 eligible articles, English-language reviewers guided by the 10 questions mistakenly judged 6 as ineligible; of the 80 ineligible articles, 8 were incorrectly judged eligible by English-language reviewers (sensitivity=0.89; specificity=0.90). Use of a simple three-step rule (excluding languages with less than three articles, reviewing titles and abstracts for clear indications of eligibility, and noting the lack of a clearly reported statistical analysis unless the word "random" appears) led to accurate classification of 51 of 53 articles (sensitivity=0.96; specificity=0.70). CONCLUSION: Our findings show promise for limiting the need for non-English-language review teams in systematic reviews with large numbers of potentially eligible non-English-language articles.
Authors: Elie A Akl; Lara A Kahale; Thomas Agoritsas; Romina Brignardello-Petersen; Jason W Busse; Alonso Carrasco-Labra; Shanil Ebrahim; Bradley C Johnston; Ignacio Neumann; Ivan Sola; Xin Sun; Per Vandvik; Yuqing Zhang; Pablo Alonso-Coello; Gordon Guyatt Journal: Syst Rev Date: 2015-07-23
Authors: Sohail M Mulla; D Norman Buckley; Dwight E Moulin; Rachel Couban; Zain Izhar; Arnav Agarwal; Akbar Panju; Li Wang; Sun Makosso Kallyth; Alparslan Turan; Victor M Montori; Daniel I Sessler; Lehana Thabane; Gordon H Guyatt; Jason W Busse Journal: BMJ Open Date: 2014-11-20 Impact factor: 2.692
Authors: Elie A Akl; Khaled Shawwa; Lara A Kahale; Thomas Agoritsas; Romina Brignardello-Petersen; Jason W Busse; Alonso Carrasco-Labra; Shanil Ebrahim; Bradley C Johnston; Ignacio Neumann; Ivan Sola; Xin Sun; Per Vandvik; Yuqing Zhang; Pablo Alonso-Coello; Gordon H Guyatt Journal: BMJ Open Date: 2015-12-30 Impact factor: 2.692
Authors: Yulia Shenderovich; Manuel Eisner; Christopher Mikton; Frances Gardner; Jianghong Liu; Joseph Murray Journal: BMC Med Res Methodol Date: 2016-03-15 Impact factor: 4.615