Laura Clark1, Caroline Fairhurst2, Catherine E Hewitt2, Yvonne Birks3, Sally Brabyn4, Sarah Cockayne2, Sara Rodgers2, Katherine Hicks2, Robert Hodgson2, Elizabeth Littlewood4, David J Torgerson2. 1. Department of Health Sciences, York Trials Unit, University of York, York YO10 5DD, United Kingdom. Electronic address: Laura.clark@york.ac.uk. 2. Department of Health Sciences, York Trials Unit, University of York, York YO10 5DD, United Kingdom. 3. Department of Social Policy, University of York, York YO10 5DD, United Kingdom. 4. Department of Health Sciences, Mental Health and Addictions Research Group, University of York, York YO10 5DD, United Kingdom.
Abstract
BACKGROUND: There is evidence to suggest that component randomized controlled trials (RCTs) within systematic reviews may be biased. It is important that these reviews are identified to prevent erroneous conclusions influencing health care policies and decisions. PURPOSE: To assess the likelihood of bias in trials in 12 meta-analyses. DESIGN: A review of 12 systematic reviews. DATA SOURCES: Twelve recently published systematic reviews with 503 component randomized trials, published in the British Medical Journal, The Lancet, Journal of the American Medical Association, and The Annals of Internal Medicine before May 2012. STUDY SELECTION AND DATA EXTRACTION: Systematic reviews were eligible for inclusion if they included only RCTs. We obtained the full text for the component RCTs of the 12 systematic reviews (in English only). We extracted summary data on age, number of participants in each treatment group, and the method of allocation concealment for each RCT. DATA SYNTHESIS: Five of the 12 meta-analyses exhibited heterogeneity in age differences (I(2) > 0.30), when there should have been none. In two meta-analyses, the age of the intervention group was significantly greater than that of the control group. Inadequate allocation concealment was a statistically significant predictor of heterogeneity in one trial as observed by a metaregression. CONCLUSIONS: Most of the sample of recent meta-analyses showed that there were signs of imbalance and/or heterogeneity in ages between treatment groups, when there should have been none. Systematic reviewers might consider using the techniques described here to assess the validity of their findings.
BACKGROUND: There is evidence to suggest that component randomized controlled trials (RCTs) within systematic reviews may be biased. It is important that these reviews are identified to prevent erroneous conclusions influencing health care policies and decisions. PURPOSE: To assess the likelihood of bias in trials in 12 meta-analyses. DESIGN: A review of 12 systematic reviews. DATA SOURCES: Twelve recently published systematic reviews with 503 component randomized trials, published in the British Medical Journal, The Lancet, Journal of the American Medical Association, and The Annals of Internal Medicine before May 2012. STUDY SELECTION AND DATA EXTRACTION: Systematic reviews were eligible for inclusion if they included only RCTs. We obtained the full text for the component RCTs of the 12 systematic reviews (in English only). We extracted summary data on age, number of participants in each treatment group, and the method of allocation concealment for each RCT. DATA SYNTHESIS: Five of the 12 meta-analyses exhibited heterogeneity in age differences (I(2) > 0.30), when there should have been none. In two meta-analyses, the age of the intervention group was significantly greater than that of the control group. Inadequate allocation concealment was a statistically significant predictor of heterogeneity in one trial as observed by a metaregression. CONCLUSIONS: Most of the sample of recent meta-analyses showed that there were signs of imbalance and/or heterogeneity in ages between treatment groups, when there should have been none. Systematic reviewers might consider using the techniques described here to assess the validity of their findings.
Authors: Tessa A Hulshof; Sytse U Zuidema; Peter J K van Meer; Christine C Gispen-de Wied; Hendrika J Luijendijk Journal: Int J Methods Psychiatr Res Date: 2018-12-04 Impact factor: 4.035
Authors: Darren J Edwards; Elinor Rainey; Vasiliki Boukouvala; Yasmin Wells; Paul Bennett; Jeremy Tree; Andrew H Kemp Journal: BMJ Open Date: 2019-07-16 Impact factor: 2.692
Authors: David Hohenschurz-Schmidt; Bethea A Kleykamp; Jerry Draper-Rodi; Jan Vollert; Jessica Chan; McKenzie Ferguson; Ewan McNicol; Jules Phalip; Scott R Evans; Dennis C Turk; Robert H Dworkin; Andrew S C Rice Journal: Pain Date: 2022-01-01 Impact factor: 6.961