BACKGROUND: Randomized controlled trials are considered the gold standard for evidence on therapeutic interventions; however, they are susceptible to bias. The objectives of this observational study were to describe the methodological quality of neonatal randomized controlled trials and quantify the bias related to specific methodological and study-level characteristics. METHODS: Twenty-five systematic reviews yielding 208 neonatal trials were included. Two independent reviewers assessed risk of bias (RoB) on seven domains consisting of nine items. For each domain, meta-analyses with at least one high/unclear and one low risk study were included in the analysis. For the primary outcome within each meta-analysis a ratio of odds ratios with a 95% confidence interval was generated. The ratio of odds ratios for each meta-analysis were combined using meta-analytic techniques with inverse-variance weighting and a random effects model to obtain a summary ratio of odds ratio. RESULTS: None of the studies had an overall low RoB. Most studies had a low RoB for the domain of incomplete outcome data (89%), while 63%, 55% and 46% of trials had low RoB for sequence generation, other sources of bias, and blinding of outcome assessors, respectively. For all other domains (allocation concealment, blinding of parents and investigators and selective outcome reporting), the majority of trials were assessed as unclear. Selective outcome reporting was rated as unclear RoB for 55% and high for 42% of studies. The only domain that showed a statistically significant association with the treatment effect was selective outcome reporting: trials at unclear/high risk of bias for this domain significantly overestimated the treatment effects compared with those assessed at low risk of bias (ROR = 1.87, 95% confidence interval: 1.26-2.78). CONCLUSIONS: This observational study of a sample of neonatal trials showed that most were at high risk of bias, indicating that there is room for improvement in the design, conduct and reporting of neonatal trials to ensure valid results for the most clinically important outcomes. We did not find an association between most risk of bias domains and effect estimates; however, we found that randomized controlled trials at high risk for selective outcome reporting were associated with overestimates of treatment benefits. These results need to be confirmed in larger samples.
BACKGROUND: Randomized controlled trials are considered the gold standard for evidence on therapeutic interventions; however, they are susceptible to bias. The objectives of this observational study were to describe the methodological quality of neonatal randomized controlled trials and quantify the bias related to specific methodological and study-level characteristics. METHODS: Twenty-five systematic reviews yielding 208 neonatal trials were included. Two independent reviewers assessed risk of bias (RoB) on seven domains consisting of nine items. For each domain, meta-analyses with at least one high/unclear and one low risk study were included in the analysis. For the primary outcome within each meta-analysis a ratio of odds ratios with a 95% confidence interval was generated. The ratio of odds ratios for each meta-analysis were combined using meta-analytic techniques with inverse-variance weighting and a random effects model to obtain a summary ratio of odds ratio. RESULTS: None of the studies had an overall low RoB. Most studies had a low RoB for the domain of incomplete outcome data (89%), while 63%, 55% and 46% of trials had low RoB for sequence generation, other sources of bias, and blinding of outcome assessors, respectively. For all other domains (allocation concealment, blinding of parents and investigators and selective outcome reporting), the majority of trials were assessed as unclear. Selective outcome reporting was rated as unclear RoB for 55% and high for 42% of studies. The only domain that showed a statistically significant association with the treatment effect was selective outcome reporting: trials at unclear/high risk of bias for this domain significantly overestimated the treatment effects compared with those assessed at low risk of bias (ROR = 1.87, 95% confidence interval: 1.26-2.78). CONCLUSIONS: This observational study of a sample of neonatal trials showed that most were at high risk of bias, indicating that there is room for improvement in the design, conduct and reporting of neonatal trials to ensure valid results for the most clinically important outcomes. We did not find an association between most risk of bias domains and effect estimates; however, we found that randomized controlled trials at high risk for selective outcome reporting were associated with overestimates of treatment benefits. These results need to be confirmed in larger samples.
Authors: Matthew J Page; Julian P T Higgins; Gemma Clayton; Jonathan A C Sterne; Asbjørn Hróbjartsson; Jelena Savović Journal: PLoS One Date: 2016-07-11 Impact factor: 3.240
Authors: Jelena Savovic; Rebecca M Turner; David Mawdsley; Hayley E Jones; Rebecca Beynon; Julian P T Higgins; Jonathan A C Sterne Journal: Am J Epidemiol Date: 2018-05-01 Impact factor: 4.897
Authors: Matthew Vassar; Sam Jellison; Hannah Wendelbo; Cole Wayant; Harrison Gray; Michael Bibens Journal: BMJ Open Date: 2019-09-06 Impact factor: 2.692
Authors: Ognjen Barcot; Matija Boric; Svjetlana Dosenovic; Marija Cavar; Antonia Jelicic Kadic; Tina Poklepovic Pericic; Ivana Vukicevic; Ivana Vuka; Livia Puljak Journal: BMC Med Res Methodol Date: 2020-09-29 Impact factor: 4.615
Authors: Kimia Honarmand; Jeremy Penn; Arnav Agarwal; Reed Siemieniuk; Romina Brignardello-Petersen; Jessica J Bartoszko; Dena Zeraatkar; Thomas Agoritsas; Karen Burns; Shannon M Fernando; Farid Foroutan; Long Ge; Francois Lamontagne; Mario A Jimenez-Mora; Srinivas Murthy; Juan Jose Yepes-Nuñez; Per O Vandvik; Zhikang Ye; Bram Rochwerg Journal: J Clin Epidemiol Date: 2021-07-15 Impact factor: 6.437