Joanna IntHout1, John P A Ioannidis2, George F Borm3, Jelle J Goeman3. 1. Radboud Institute for Health Sciences (RIHS), Radboud University Medical Center, Mailbox 133, P.O. Box 9101, Nijmegen 6500 HB, The Netherlands. Electronic address: joanna.inthout@radboudumc.nl. 2. Stanford Prevention Research Center, Department of Medicine, Stanford University School of Humanities and Sciences, 1265 Welch Road, Stanford, CA 94305, USA; Department of Health Research and Policy, Stanford University School of Medicine, 150 Governor's Lane, Stanford, CA 94305, USA; Department of Statistics, Stanford University School of Humanities and Sciences, 390 Serra Mall, Stanford, CA 94305, USA; Meta-Research Innovation Center at Stanford (METRICS), Stanford University, 1265 Welch Road, Stanford, CA 94305, USA. 3. Radboud Institute for Health Sciences (RIHS), Radboud University Medical Center, Mailbox 133, P.O. Box 9101, Nijmegen 6500 HB, The Netherlands.
Abstract
OBJECTIVES: Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). STUDY DESIGN AND SETTING: We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. RESULTS: Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. CONCLUSION: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.
OBJECTIVES: Between-study heterogeneity plays an important role in random-effects models for meta-analysis. Most clinical trials are small, and small trials are often associated with larger effect sizes. We empirically evaluated whether there is also a relationship between trial size and heterogeneity (τ). STUDY DESIGN AND SETTING: We selected the first meta-analysis per intervention review of the Cochrane Database of Systematic Reviews Issues 2009-2013 with a dichotomous (n = 2,009) or continuous (n = 1,254) outcome. The association between estimated τ and trial size was evaluated across meta-analyses using regression and within meta-analyses using a Bayesian approach. Small trials were predefined as those having standard errors (SEs) over 0.2 standardized effects. RESULTS: Most meta-analyses were based on few (median 4) trials. Within the same meta-analysis, the small study τS(2) was larger than the large-study τL(2) [average ratio 2.11; 95% credible interval (1.05, 3.87) for dichotomous and 3.11 (2.00, 4.78) for continuous meta-analyses]. The imprecision of τS was larger than of τL: median SE 0.39 vs. 0.20 for dichotomous and 0.22 vs. 0.13 for continuous small-study and large-study meta-analyses. CONCLUSION: Heterogeneity between small studies is larger than between larger studies. The large imprecision with which τ is estimated in a typical small-studies' meta-analysis is another reason for concern, and sensitivity analyses are recommended.
Authors: Philip J Wiffen; Sheena Derry; R Andrew Moore; Ewan D McNicol; Rae F Bell; Daniel B Carr; Mairead McIntyre; Bee Wee Journal: Cochrane Database Syst Rev Date: 2017-07-12
Authors: Sheena Derry; Philip J Wiffen; R Andrew Moore; Ewan D McNicol; Rae F Bell; Daniel B Carr; Mairead McIntyre; Bee Wee Journal: Cochrane Database Syst Rev Date: 2017-07-12
Authors: Julio A Yanes; Zach E McKinnell; Meredith A Reid; Jessica N Busler; Jesse S Michel; Melissa M Pangelinan; Matthew T Sutherland; Jarred W Younger; Raul Gonzalez; Jennifer L Robinson Journal: Exp Clin Psychopharmacol Date: 2019-05-23 Impact factor: 3.157
Authors: Louise J Geneen; R Andrew Moore; Clare Clarke; Denis Martin; Lesley A Colvin; Blair H Smith Journal: Cochrane Database Syst Rev Date: 2017-04-24