S C Hay1,2, H Kirpalani3, C Viner4, R Soll5, D Dukhovny6, W-Y Mao1, J Profit7, S B DeMauro3, J A F Zupancic1,2. 1. Department of Neonatology, Beth Israel Deaconess Medical Center, Boston, MA, USA. 2. Division of Newborn Medicine, Boston Children's Hospital, Harvard Medical School, Boston, MA, USA. 3. Division of Neonatology, Children's Hospital of Philadelphia, Philadelphia, PA, USA. 4. Department of Obstetrics and Gynecology, New York University, New York, NY, USA. 5. Division of Neonatal Perinatal Medicine, University of Vermont College of Medicine, Burlington, VT, USA. 6. Division of Neonatology, Oregon Health and Science University, Portland, OR, USA. 7. Division of Neonatology, Stanford University, Stanford, CA, USA.
Abstract
OBJECTIVE: To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews. STUDY DESIGN: We selected meta-analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact. RESULTS: For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ⩾20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference. CONCLUSION: Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.
OBJECTIVE: To assess the impact of the latest randomized controlled trial (RCT) to each systematic review (SR) in Cochrane Neonatal Reviews. STUDY DESIGN: We selected meta-analyses reporting the typical point estimate of the risk ratio for the primary outcome of the latest study (n=130), mortality (n=128) and the mean difference for the primary outcome (n=44). We employed cumulative meta-analysis to determine the typical estimate after each trial was added, and then performed multivariable logistic regression to determine factors predictive of study impact. RESULTS: For the stated primary outcome, 18% of latest RCTs failed to narrow the confidence interval (CI), and 55% failed to decrease the CI by ⩾20%. Only 8% changed the typical estimate directionality, and 11% caused a change to or from significance. Latest RCTs did not change the typical estimate in 18% of cases, and only 41% changed the typical estimate by at least 10%. The ability to narrow the CI by >20% was negatively associated with the number of previously published RCTs (odds ratio 0.707). Similar results were found in analysis of typical estimates for the outcomes of mortality and mean difference. CONCLUSION: Across a broad range of clinical questions, the latest RCT failed to substantially narrow the CI of the typical estimate, to move the effect estimate or to change its statistical significance in a majority of cases. Investigators and grant peer review committees should consider prioritizing less-studied topics or requiring formal consideration of optimal information size based on extant evidence in power calculations.
Authors: John C Sinclair; Diane E Haughton; Michael B Bracken; Jeffrey D Horbar; Roger F Soll Journal: Clin Perinatol Date: 2003-06 Impact factor: 3.430
Authors: Katherine S Button; John P A Ioannidis; Claire Mokrysz; Brian A Nosek; Jonathan Flint; Emma S J Robinson; Marcus R Munafò Journal: Nat Rev Neurosci Date: 2013-04-10 Impact factor: 34.870
Authors: David Gordon; Wendy Taddei-Peters; Alice Mascette; Melissa Antman; Peter G Kaufmann; Michael S Lauer Journal: N Engl J Med Date: 2013-11-14 Impact factor: 91.245