Juliana Martins Scalabrin1, Marcelo F Mello1, Walter Swardfager2,3, Hugo Cogo-Moreira1. 1. Department of Psychiatry, Universidade Federal de Sao Paulo, Sao Paulo, Brazil. 2. Department of Pharmacology and Toxicology, University of Toronto, Toronto, Ontario, Canada. 3. Hurvitz Brain Sciences Program, Sunnybrook Research Institute, Toronto, Ontario, Canada.
Abstract
OBJECTIVE: To evaluate the factorial validity and internal consistency of a measurement model underlying risk of bias as endorsed by Cochrane for use in systematic reviews; more specifically, how the risk of bias tool behaves in the context of studies on psychological therapies used for treatment of post-traumatic stress disorder in adults. METHODS: We applied confirmatory factor analysis to a systematic review containing 70 clinical trials entitled "Psychological Therapies for Chronic Post-Traumatic Stress Disorder in Adults" under a Bayesian estimator. Seven observed categorical risk of bias items (answered categorically as low, unclear, or high risk of bias) were collected from the systematic review. RESULTS: A unidimensional model for the Cochrane risk of bias tool items returned poor fit indices and low factor loadings, indicating questionable validity and internal consistency. CONCLUSION: Although the present evidence is restricted to psychological interventions for post-traumatic stress disorder, it demonstrates that the way risk of bias has been measured in this context may not be adequate. More broadly, the results suggest the importance of testing the risk of bias tool, and the possibility of rethinking the methods used to assess risk of bias in systematic reviews and meta-analyses.
OBJECTIVE: To evaluate the factorial validity and internal consistency of a measurement model underlying risk of bias as endorsed by Cochrane for use in systematic reviews; more specifically, how the risk of bias tool behaves in the context of studies on psychological therapies used for treatment of post-traumatic stress disorder in adults. METHODS: We applied confirmatory factor analysis to a systematic review containing 70 clinical trials entitled "Psychological Therapies for Chronic Post-Traumatic Stress Disorder in Adults" under a Bayesian estimator. Seven observed categorical risk of bias items (answered categorically as low, unclear, or high risk of bias) were collected from the systematic review. RESULTS: A unidimensional model for the Cochrane risk of bias tool items returned poor fit indices and low factor loadings, indicating questionable validity and internal consistency. CONCLUSION: Although the present evidence is restricted to psychological interventions for post-traumatic stress disorder, it demonstrates that the way risk of bias has been measured in this context may not be adequate. More broadly, the results suggest the importance of testing the risk of bias tool, and the possibility of rethinking the methods used to assess risk of bias in systematic reviews and meta-analyses.