Gerald Gartlehner1, Barbara Nussbaumer-Streit2, Gernot Wagner2, Sheila Patel3, Tammeka Swinson-Evans3, Andreea Dobrescu4, Christian Gluud5. 1. RTI International, Research Triangle Park, NC, USA; Cochrane Austria, Danube University Krems, Krems, Austria. Electronic address: ggartlehner@rti.org. 2. Cochrane Austria, Danube University Krems, Krems, Austria. 3. RTI International, Research Triangle Park, NC, USA. 4. Genetics Department, Victor Babes University of Medicine and Pharmacy, Timisoara, Romania. 5. Copenhagen Trial Unit; Centre for Clinical Intervention Research, Rigshospitalet, Copenhagen University Hospital, Copenhagen, Denmark.
Abstract
OBJECTIVES: The aim of article was to assess the risk for random errors in outcomes graded as high certainty of evidence (CoE). STUDY DESIGN AND SETTING: We randomly selected 100 Cochrane reviews with dichotomous outcomes rated as high CoE using Grading of Recommendations Assessment, Development, and Evaluation. To detect increased risks for random errors, two investigators independently conducted trial sequential analysis using conventional thresholds for type I (α = 0.05) and type II (β = 0.10) errors. We dually regraded all outcomes with increased risks for random errors and conducted multivariate logistic regression analyses to determine predictors of increased risks for random errors. RESULTS: Overall, 38% (95% confidence interval: 28-47%) of high CoE outcomes had increased risks for random errors. Outcomes assessing harms were more frequently affected than outcomes assessing benefits (47% vs. 12%). Regrading of outcomes with increased random errors showed that 74% should have been downgraded based on current guidance. Regression analyses rendered small absolute risk differences (P = 0.009) and low number of events (P = 0.001) as significant predictors of increased risks for random errors. CONCLUSION: Decisionmakers need to be aware that outcomes rated as high CoE often have increased risks for false-positive or false-negative findings.
OBJECTIVES: The aim of article was to assess the risk for random errors in outcomes graded as high certainty of evidence (CoE). STUDY DESIGN AND SETTING: We randomly selected 100 Cochrane reviews with dichotomous outcomes rated as high CoE using Grading of Recommendations Assessment, Development, and Evaluation. To detect increased risks for random errors, two investigators independently conducted trial sequential analysis using conventional thresholds for type I (α = 0.05) and type II (β = 0.10) errors. We dually regraded all outcomes with increased risks for random errors and conducted multivariate logistic regression analyses to determine predictors of increased risks for random errors. RESULTS: Overall, 38% (95% confidence interval: 28-47%) of high CoE outcomes had increased risks for random errors. Outcomes assessing harms were more frequently affected than outcomes assessing benefits (47% vs. 12%). Regrading of outcomes with increased random errors showed that 74% should have been downgraded based on current guidance. Regression analyses rendered small absolute risk differences (P = 0.009) and low number of events (P = 0.001) as significant predictors of increased risks for random errors. CONCLUSION: Decisionmakers need to be aware that outcomes rated as high CoE often have increased risks for false-positive or false-negative findings.
Authors: Ning Liang; De Zhao Kong; Si Si Ma; Chun Li Lu; Ming Yang; Lu Da Feng; Chen Shen; Ruo Han Diao; Ling Jun Cui; Xing Yu Lu; Dimitrinka Nikolova; Janus C Jakobsen; Christian Gluud; Jian Ping Liu Journal: Cochrane Database Syst Rev Date: 2019-04-03
Authors: De Zhao Kong; Ning Liang; Guan Lin Yang; Zhe Zhang; Yue Liu; Ye Yang; Yu Xi Liu; Qi Ge Wang; Fan Zhang; Hui Yong Zhang; Dimitrinka Nikolova; Janus C Jakobsen; Christian Gluud; Jian Ping Liu Journal: Cochrane Database Syst Rev Date: 2019-08-22
Authors: Ning Liang; De Zhao Kong; Chun Li Lu; Si Si Ma; Yu Qi Li; Dimitrinka Nikolova; Janus C Jakobsen; Christian Gluud; Jian Ping Liu Journal: Cochrane Database Syst Rev Date: 2019-06-24
Authors: Rosa G Simonetti; Giovanni Perricone; Dimitrinka Nikolova; Goran Bjelakovic; Christian Gluud Journal: Cochrane Database Syst Rev Date: 2019-06-28
Authors: Rosa G Simonetti; Giovanni Perricone; Helen L Robbins; Narendra R Battula; Martin O Weickert; Robert Sutton; Saboor Khan Journal: Cochrane Database Syst Rev Date: 2020-10-22