Richard D Riley1, Simon Gates, James Neilson, Zarko Alfirevic. 1. Department of Public Health, Epidemiology and Biostatistics, Public Health Building, University of Birmingham, Edgbaston, Birmingham B15 2TT, UK. r.d.riley@bham.ac.uk
Abstract
OBJECTIVES: To assess statistical methods within systematic reviews of the Cochrane Pregnancy and Childbirth Group (CPCG). STUDY DESIGN AND SETTING: We extracted details about statistical methods within 75 reviews containing at least 10 studies. RESULTS: The median number of forest plots per review was 52 (min=5; max=409). Seven of the 75 reviews assessed publication bias or explained why not. Forty-four of the 75 reviews performed random-effects meta-analyses; just 1 of these justified the approach clinically and none interpreted its pooled result correctly. Of 31 reviews not using random-effects, 26 assumed a fixed-effect given potentially moderate or large heterogeneity (I(2)>25%). In their Methods section, 25 (33%) of the 75 reviews said I(2) was used to decide between fixed-effect and random-effects; however, in 12 of these (48%) reviews, this was not carried out in their Results section. Of 72 reviews with moderate or large heterogeneity, 47 (65%) did not explore the causes of heterogeneity or justify why not. CONCLUSION: Within CPCG reviews, publication bias is rarely addressed; heterogeneity is often not appropriately considered, and random-effects analyses are incorrectly interpreted. How these shortcomings impact existing review conclusions needs further investigation, but regardless of this, we recomment the Cochrane Collaboration increase "hands-on" statistical support.
OBJECTIVES: To assess statistical methods within systematic reviews of the Cochrane Pregnancy and Childbirth Group (CPCG). STUDY DESIGN AND SETTING: We extracted details about statistical methods within 75 reviews containing at least 10 studies. RESULTS: The median number of forest plots per review was 52 (min=5; max=409). Seven of the 75 reviews assessed publication bias or explained why not. Forty-four of the 75 reviews performed random-effects meta-analyses; just 1 of these justified the approach clinically and none interpreted its pooled result correctly. Of 31 reviews not using random-effects, 26 assumed a fixed-effect given potentially moderate or large heterogeneity (I(2)>25%). In their Methods section, 25 (33%) of the 75 reviews said I(2) was used to decide between fixed-effect and random-effects; however, in 12 of these (48%) reviews, this was not carried out in their Results section. Of 72 reviews with moderate or large heterogeneity, 47 (65%) did not explore the causes of heterogeneity or justify why not. CONCLUSION: Within CPCG reviews, publication bias is rarely addressed; heterogeneity is often not appropriately considered, and random-effects analyses are incorrectly interpreted. How these shortcomings impact existing review conclusions needs further investigation, but regardless of this, we recomment the Cochrane Collaboration increase "hands-on" statistical support.
Authors: Miriam Hattle; Danielle L Burke; Thomas Trikalinos; Christopher H Schmid; Yong Chen; Dan Jackson; Richard D Riley Journal: Syst Rev Date: 2022-07-26
Authors: Malcolm J Price; Helen A Blake; Sara Kenyon; Ian R White; Dan Jackson; Jamie J Kirkham; James P Neilson; Jonathan J Deeks; Richard D Riley Journal: Res Synth Methods Date: 2019-08-12 Impact factor: 5.273