B Nussbaumer-Streit1, I Klerings2, A I Dobrescu2, E Persad2, A Stevens3, C Garritty3, C Kamel4, L Affengruber5, V J King6, G Gartlehner7. 1. Cochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems a.d. Donau, Austria. Electronic address: barbara.nussbaumer-streit@donau-uni.ac.at. 2. Cochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems a.d. Donau, Austria. 3. Knowledge Synthesis Group, Clinical Epidemiology Program, Ottawa Hospital Research Institute, Ottawa, Canada. 4. HTA and Rapid Response, Canadian Agency for Drugs and Technologies in Health (CADTH), Ottawa, Canada. 5. Cochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems a.d. Donau, Austria; Department of Family Medicine, Care and Public Health Research Institute (CAPHRI), Maastricht University, The Netherlands. 6. The Center for Evidence-based Policy, Oregon Health & Science University, Portland, Oregon, United States. 7. Cochrane Austria, Department for Evidence-based Medicine and Evaluation, Danube University Krems, Krems a.d. Donau, Austria; RTI International, Research Triangle Park, NC, USA.
Abstract
OBJECTIVES: We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses. STUDY DESIGN AND SETTING: Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%. RESULTS: Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study. Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small. CONCLUSION: Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews.
OBJECTIVES: We aimed to assess whether limiting the inclusion criteria solely to English-language publications affected the overall conclusions of evidence syntheses. STUDY DESIGN AND SETTING: Our analyses used a dataset of a previous methods study that included 59 randomly selected Cochrane intervention reviews with no language restrictions. First, we ascertained the publication language of all 2,026 included publications. Next, we excluded studies based on the following criteria: (1) publication solely in non-English language, or (2) main publication (in case of multiple publications of the same study) in non-English language. We then re-calculated meta-analyses for outcomes that were presented in the main summary of findings tables of the Cochrane reports. If the direction of the effect estimate or the statistical significance changed, authors of the respective Cochrane reviews were consulted to assess whether the new evidence base would have changed their conclusions. The primary outcome of our analyses examined the proportion of conclusions that would change with the exclusion of non-English publications. We set the threshold for the approach as noninferior if the upper limit of the 95% confidence interval of the proportion of changed conclusions did not cross a margin of 10%. RESULTS: Across all 59 Cochrane reviews, 29 (49%) included 80 non-English publications. For 16 (27%) of these Cochrane reviews, the exclusion of non-English publications resulted in the exclusion of at least one study. In the remaining 13 Cochrane reviews, the non-English publications were not the only or main publication of the study or they did not contribute to the main summary of the findings table, so their exclusion did not result in an exclusion of the study. Overall, the exclusion of non-English publications led to the exclusion of 31 studies contributing to 40 outcomes. For 38 of the 40 outcomes, the exclusion of non-English studies did not markedly alter the size or direction of effect estimates or statistical significance. In two outcomes, the statistical significance changed, but authors would have still drawn the same conclusion, albeit with less certainty. Thus, the proportion of changed conclusions in our sample was 0.0% (95% CI 0.0-0.6), which indicated the noninferiority of the approach. However, the majority of excluded studies were small. CONCLUSION: Exclusion of non-English publications from systematic reviews on clinical interventions had a minimal effect on overall conclusions and could be a viable methodological shortcut, especially for rapid reviews.
Authors: Jing Zhang; Chunyan Li; Junjie Xu; Zhili Hu; Sarah E Rutstein; Joseph D Tucker; Jason J Ong; Yongjun Jiang; Wenqing Geng; Sarah T Wright; Myron S Cohen; Hong Shang; Weiming Tang Journal: Lancet HIV Date: 2022-04 Impact factor: 16.070
Authors: Travis J Saunders; Travis McIsaac; Jenny Campbell; Kevin Douillette; Ian Janssen; Jennifer R Tomasone; Amanda Ross-White; Stephanie A Prince; Jean-Philippe Chaput Journal: Health Promot Chronic Dis Prev Can Date: 2022-04 Impact factor: 2.725
Authors: Caroline Dutil; Irina Podinic; Christin M Sadler; Bruno G da Costa; Ian Janssen; Amanda Ross-White; Travis J Saunders; Jennifer R Tomasone; Jean-Philippe Chaput Journal: Health Promot Chronic Dis Prev Can Date: 2022-04 Impact factor: 2.725
Authors: Ian Janssen; Julie E Campbell; Samah Zahran; Travis J Saunders; Jennifer R Tomasone; Jean-Philippe Chaput Journal: Health Promot Chronic Dis Prev Can Date: 2022-04 Impact factor: 2.725
Authors: Manu Shankar-Hari; Rohit Saha; Julie Wilson; Hallie C Prescott; David Harrison; Kathryn Rowan; Gordon D Rubenfeld; Neill K J Adhikari Journal: Intensive Care Med Date: 2020-01-23 Impact factor: 17.440