Peter Herbison1, Jean Hay-Smith, William J Gillespie. 1. Department of Preventive and Social Medicine, Dunedin School of Medicine, University of Otago, PO Box 913, Dunedin, New Zealand. peter.herbison@otago.ac.nz
Abstract
OBJECTIVE: To find if a particular quality score was better than others at validly scoring the quality of randomized controlled trials, both by examining the consistency of dividing studies into high and low quality and using a large study as a reference standard. STUDY DESIGN AND SETTING: Observational study of meta-analyses from the Cochrane Library. These had to have binary outcomes that included more than 10 studies, one or more of which randomized more than 500 people into each group. RESULTS: Eighteen systematic reviews, with 65 meta-analyses using binary outcomes, were included and the included trials were scored for 43 different quality scores. None of these scores was better at dividing the studies in to low and high quality, and none of the scores was better over the 65 meta-analyses in making the result closer to the reference standard. CONCLUSION: None of the quality scores found appeared to measure quality validly. It is a mistake to assign meaning to the result of a quality score.
OBJECTIVE: To find if a particular quality score was better than others at validly scoring the quality of randomized controlled trials, both by examining the consistency of dividing studies into high and low quality and using a large study as a reference standard. STUDY DESIGN AND SETTING: Observational study of meta-analyses from the Cochrane Library. These had to have binary outcomes that included more than 10 studies, one or more of which randomized more than 500 people into each group. RESULTS: Eighteen systematic reviews, with 65 meta-analyses using binary outcomes, were included and the included trials were scored for 43 different quality scores. None of these scores was better at dividing the studies in to low and high quality, and none of the scores was better over the 65 meta-analyses in making the result closer to the reference standard. CONCLUSION: None of the quality scores found appeared to measure quality validly. It is a mistake to assign meaning to the result of a quality score.
Authors: Michael Roerecke; Afshin Vafaei; Omer S M Hasan; Bethany R Chrystoja; Marcus Cruz; Roy Lee; Manuela G Neuman; Jürgen Rehm Journal: Am J Gastroenterol Date: 2019-10 Impact factor: 10.864
Authors: Susan Armijo-Olivo; Jorge Fuentes; Maria Ospina; Humam Saltaji; Lisa Hartling Journal: BMC Med Res Methodol Date: 2013-09-17 Impact factor: 4.615
Authors: Tatyana A Shamliyan; Jae-Young Choi; Rema Ramakrishnan; Jennifer Biggs Miller; Shi-Yi Wang; Frederick R Taylor; Robert L Kane Journal: J Gen Intern Med Date: 2013-04-17 Impact factor: 5.128
Authors: Cosetta Minelli; John R Thompson; Keith R Abrams; Ammarin Thakkinstian; John Attia Journal: Am J Epidemiol Date: 2009-11-09 Impact factor: 4.897