RATIONALE: Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically ill patients commonly fail to detect treatment benefits. OBJECTIVES: Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. METHODS: One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. MEASUREMENT AND MAIN RESULTS: Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. CONCLUSIONS: ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficient patient accrual, and incorrect predictions of baseline event rates.
RATIONALE: Intensive care unit (ICU)-based randomized clinical trials (RCTs) among adult critically illpatients commonly fail to detect treatment benefits. OBJECTIVES: Appraise the rates of success, outcomes used, statistical power, and design characteristics of published trials. METHODS: One hundred forty-six ICU-based RCTs of diagnostic, therapeutic, or process/systems interventions published from January 2007 to May 2013 in 16 high-impact general or critical care journals were studied. MEASUREMENT AND MAIN RESULTS: Of 146 RCTs, 54 (37%) were positive (i.e., the a priori hypothesis was found to be statistically significant). The most common primary outcomes were mortality (n = 40 trials), infection-related outcomes (n = 33), and ventilation-related outcomes (n = 30), with positive results found in 10, 58, and 43%, respectively. Statistical power was discussed in 135 RCTs (92%); 92 cited a rationale for their power parameters. Twenty trials failed to achieve at least 95% of their reported target sample size, including 11 that were stopped early due to insufficient accrual/logistical issues. Of 34 superiority RCTs comparing mortality between treatment arms, 13 (38%) accrued a sample size large enough to find an absolute mortality reduction of 10% or less. In 22 of these trials the observed control-arm mortality rate differed from the predicted rate by at least 7.5%. CONCLUSIONS: ICU-based RCTs are commonly negative and powered to identify what appear to be unrealistic treatment effects, particularly when using mortality as the primary outcome. Additional concerns include a lack of standardized methods for assessing common outcomes, unclear justifications for statistical power calculations, insufficientpatient accrual, and incorrect predictions of baseline event rates.
Authors: Paul A Harris; Robert Taylor; Robert Thielke; Jonathon Payne; Nathaniel Gonzalez; Jose G Conde Journal: J Biomed Inform Date: 2008-09-30 Impact factor: 6.317
Authors: John A Myburgh; Simon Finfer; Rinaldo Bellomo; Laurent Billot; Alan Cass; David Gattas; Parisa Glass; Jeffrey Lipman; Bette Liu; Colin McArthur; Shay McGuinness; Dorrilyn Rajbhandari; Colman B Taylor; Steven A R Webb Journal: N Engl J Med Date: 2012-10-17 Impact factor: 91.245
Authors: Niall D Ferguson; Damon C Scales; Ruxandra Pinto; M Elizabeth Wilcox; Deborah J Cook; Gordon H Guyatt; Holger J Schünemann; John C Marshall; Margaret S Herridge; Maureen O Meade Journal: Am J Respir Crit Care Med Date: 2012-11-29 Impact factor: 21.405
Authors: Timothy D Girard; John P Kress; Barry D Fuchs; Jason W W Thomason; William D Schweickert; Brenda T Pun; Darren B Taichman; Jan G Dunn; Anne S Pohlman; Paul A Kinniry; James C Jackson; Angelo E Canonico; Richard W Light; Ayumi K Shintani; Jennifer L Thompson; Sharon M Gordon; Jesse B Hall; Robert S Dittus; Gordon R Bernard; E Wesley Ely Journal: Lancet Date: 2008-01-12 Impact factor: 79.321
Authors: Bronagh Blackwood; Mike Clarke; Danny F McAuley; Peter J McGuigan; John C Marshall; Louise Rose Journal: Am J Respir Crit Care Med Date: 2014-04-15 Impact factor: 21.405
Authors: Theodore J Iwashyna; James F Burke; Jeremy B Sussman; Hallie C Prescott; Rodney A Hayward; Derek C Angus Journal: Am J Respir Crit Care Med Date: 2015-11-01 Impact factor: 21.405
Authors: Anders Perner; Anthony C Gordon; Derek C Angus; Francois Lamontagne; Flavia Machado; James A Russell; Jean-Francois Timsit; John C Marshall; John Myburgh; Manu Shankar-Hari; Mervyn Singer Journal: Intensive Care Med Date: 2017-05-12 Impact factor: 17.440
Authors: Michael E Detsky; Michael O Harhay; Dominique F Bayard; Aaron M Delman; Anna E Buehler; Saida A Kent; Isabella V Ciuffetelli; Elizabeth Cooney; Nicole B Gabler; Sarah J Ratcliffe; Mark E Mikkelsen; Scott D Halpern Journal: Ann Am Thorac Soc Date: 2017-10
Authors: Kusum Menon; James Dayre McNally; Jerry J Zimmerman; Michael S D Agus; Katie O'Hearn; R Scott Watson; Hector R Wong; Mark Duffett; David Wypij; Karen Choong Journal: Pediatr Crit Care Med Date: 2017-03 Impact factor: 3.624