Ariel M Aloe1, Betsy Jane Becker2, Maren Duvendack3, Jeffrey C Valentine4, Ian Shemilt5, Hugh Waddington6. 1. Educational Measurement and Statistics, 368 Lindquist Center, University of Iowa, Iowa City, IA 52242, USA. Electronic address: ariel-aloe@uiowa.edu. 2. College of Education, Florida State University, 3210D Stone Building, Tallahassee, FL 32306-4453, USA. 3. School of International Development, University of East Anglia, Norwich NR4 7TJ, UK. 4. Department of Counseling and Human Development, University of Louisville, 309 Porter Education Building, Louisville, KY 40292, USA. 5. EPPI-Centre, University College London, 10 Woburn Square, London WC1H 0NR, UK. 6. International Initiative for Impact Evaluation (3ie), 36 Gordon Square, London WC1H 0PD, UK.
Abstract
OBJECTIVE: To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs. STUDY DESIGN AND SETTING: All quasi-experimental (QE) designs. RESULTS: When designing a systematic review of QE studies, potential sources of heterogeneity-both theory-based and methodological-must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls used are viewed as of greatest importance. Potential sources of bias and confounding are also addressed. CONCLUSION: Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis.
OBJECTIVE: To identify variables that must be coded when synthesizing primary studies that use quasi-experimental designs. STUDY DESIGN AND SETTING: All quasi-experimental (QE) designs. RESULTS: When designing a systematic review of QE studies, potential sources of heterogeneity-both theory-based and methodological-must be identified. We outline key components of inclusion criteria for syntheses of quasi-experimental studies. We provide recommendations for coding content-relevant and methodological variables and outlined the distinction between bivariate effect sizes and partial (i.e., adjusted) effect sizes. Designs used and controls used are viewed as of greatest importance. Potential sources of bias and confounding are also addressed. CONCLUSION: Careful consideration must be given to inclusion criteria and the coding of theoretical and methodological variables during the design phase of a synthesis of quasi-experimental studies. The success of the meta-regression analysis relies on the data available to the meta-analyst. Omission of critical moderator variables (i.e., effect modifiers) will undermine the conclusions of a meta-analysis.
Authors: Julian P T Higgins; José A López-López; Betsy J Becker; Sarah R Davies; Sarah Dawson; Jeremy M Grimshaw; Luke A McGuinness; Theresa H M Moore; Eva A Rehfuess; James Thomas; Deborah M Caldwell Journal: BMJ Glob Health Date: 2019-01-25