Asbjørn Hróbjartsson1, Frida Emanuelsson2, Ann Sofia Skou Thomsen2, Jørgen Hilden2, Stig Brorson2. 1. Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and Department of Orthopaedic Surgery, Herlev University Hospital, Herlev, Denmark ah@cochrane.dk. 2. Nordic Cochrane Centre, Rigshospitalet, Copenhagen, Denmark, Department of Biostatistics, University of Copenhagen, Copenhagen, Denmark and Department of Orthopaedic Surgery, Herlev University Hospital, Herlev, Denmark.
Abstract
BACKGROUND: Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown. METHODS: Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity. RESULTS: Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43). CONCLUSIONS: This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.
BACKGROUND: Blinding patients in clinical trials is a key methodological procedure, but the expected degree of bias due to nonblinded patients on estimated treatment effects is unknown. METHODS: Systematic review of randomized clinical trials with one sub-study (i.e. experimental vs control) involving blinded patients and another, otherwise identical, sub-study involving nonblinded patients. Within each trial, we compared the difference in effect sizes (i.e. standardized mean differences) between the sub-studies. A difference <0 indicates that nonblinded patients generated a more optimistic effect estimate. We pooled the differences with random-effects inverse variance meta-analysis, and explored reasons for heterogeneity. RESULTS: Our main analysis included 12 trials (3869 patients). The average difference in effect size for patient-reported outcomes was -0.56 (95% confidence interval -0.71 to -0.41), (I(2)=60%, P=0.004), i.e. nonblinded patients exaggerated the effect size by an average of 0.56 standard deviation, but with considerable variation. Two of the 12 trials also used observer-reported outcomes, showing no indication of exaggerated effects due lack of patient blinding. There was a larger effect size difference in 10 acupuncture trials [-0.63 (-0.77 to -0.49)], than in the two non-acupuncture trials [-0.17 (-0.41 to 0.07)]. Lack of patient blinding also increased attrition and use of co-interventions: ratio of control group attrition risk 1.79 (1.18 to 2.70), and ratio of control group co-intervention risk 1.55 (0.99 to 2.43). CONCLUSIONS: This study provides empirical evidence of pronounced bias due to lack of patient blinding in complementary/alternative randomized clinical trials with patient-reported outcomes.
Authors: Kevin E Thorpe; Merrick Zwarenstein; Andrew D Oxman; Shaun Treweek; Curt D Furberg; Douglas G Altman; Sean Tunis; Eduardo Bergel; Ian Harvey; David J Magid; Kalipso Chalkidou Journal: J Clin Epidemiol Date: 2009-05 Impact factor: 6.437
Authors: G J Van Der Heijden; P Leffers; P J Wolters; J J Verheijden; H van Mameren; J P Houben; L M Bouter; P G Knipschild Journal: Ann Rheum Dis Date: 1999-09 Impact factor: 19.103
Authors: Eveline Nüesch; Stephan Reichenbach; Sven Trelle; Anne W S Rutjes; Katharina Liewald; Rebekka Sterchi; Douglas G Altman; Peter Jüni Journal: Arthritis Rheum Date: 2009-12-15
Authors: Gerdine A J Fransen; Corine J van Marrewijk; Suhreta Mujakovic; Jean W M Muris; Robert J F Laheij; Mattijs E Numans; Niek J de Wit; Melvin Samsom; Jan B M J Jansen; J André Knottnerus Journal: BMC Med Res Methodol Date: 2007-04-23 Impact factor: 4.615
Authors: Vincent C H Chung; Robin S T Ho; Siya Liu; Marc K C Chong; Albert W N Leung; Benjamin H K Yip; Sian M Griffiths; Benny C Y Zee; Justin C Y Wu; Regina W S Sit; Alexander Y L Lau; Samuel Y S Wong Journal: CMAJ Date: 2016-06-06 Impact factor: 8.262
Authors: Mateusz J Swierz; Dawid Storman; Robert P Riemsma; Robert Wolff; Jerzy W Mitus; Michal Pedziwiatr; Jos Kleijnen; Malgorzata M Bala Journal: Cochrane Database Syst Rev Date: 2020-03-12