James Cowan1, Santiago Lozano-Calderón, David Ring. 1. Orthopaedic Hand and Upper Extremity Service, Massachusetts General Hospital, Yawkey Center 2100, 55 Fruit Street, Boston, MA 02114, USA.
Abstract
BACKGROUND: The Oxford Levels of Evidence are now routinely assigned at many orthopaedic journals. One disadvantage of this approach is that study designs with a higher level of evidence may be given greater weight than the overall quality of the study merits. In other words, there is no guarantee that research is scientifically valid simply because a more sophisticated study design was employed. The aim of this study was to review Level-I and II therapeutic studies on lateral epicondylitis to measure variation in quality among the highest-level study designs. METHODS: Fifty-four prospective randomized therapeutic trials involving patients with lateral epicondylitis were evaluated by two independent reviewers according to the Oxford Levels of Evidence, a modification of the Coleman Methodology Score (a 0 to 100-point scale), and the revised CONSORT (Consolidated Standards of Reporting Trials) score. RESULTS: The two reviewers were consistent in their use of the Oxford Levels of Evidence (kappa = 0.73, p < 0.01), the modified Coleman Methodology Score (kappa = 0.73; p < 0.01), and the CONSORT score (kappa = 0.53; p < 0.01). Both reviewers rated the majority of studies as Level II (91% and 94%) and as unsatisfactory according to the Coleman Methodology Score (87% and 89%) and the CONSORT score (62% and 63%). Areas of deficiency included poor descriptions of recruitment (>90% of the trials), power-level calculations (73%), randomization (58%), blinding (90%), and participant flow (50%) as well as inadequate follow-up, sample size, and blinding. CONCLUSIONS: The use of the gold-standard trial design, the prospective randomized therapeutic study (Level-I or II evidence), does not ensure quality research or reporting. Critical analysis of scientific work is important regardless of the study design. Clinical scientists should be familiar with the CONSORT criteria and adhere to them when reporting clinical trials.
BACKGROUND: The Oxford Levels of Evidence are now routinely assigned at many orthopaedic journals. One disadvantage of this approach is that study designs with a higher level of evidence may be given greater weight than the overall quality of the study merits. In other words, there is no guarantee that research is scientifically valid simply because a more sophisticated study design was employed. The aim of this study was to review Level-I and II therapeutic studies on lateral epicondylitis to measure variation in quality among the highest-level study designs. METHODS: Fifty-four prospective randomized therapeutic trials involving patients with lateral epicondylitis were evaluated by two independent reviewers according to the Oxford Levels of Evidence, a modification of the Coleman Methodology Score (a 0 to 100-point scale), and the revised CONSORT (Consolidated Standards of Reporting Trials) score. RESULTS: The two reviewers were consistent in their use of the Oxford Levels of Evidence (kappa = 0.73, p < 0.01), the modified Coleman Methodology Score (kappa = 0.73; p < 0.01), and the CONSORT score (kappa = 0.53; p < 0.01). Both reviewers rated the majority of studies as Level II (91% and 94%) and as unsatisfactory according to the Coleman Methodology Score (87% and 89%) and the CONSORT score (62% and 63%). Areas of deficiency included poor descriptions of recruitment (>90% of the trials), power-level calculations (73%), randomization (58%), blinding (90%), and participant flow (50%) as well as inadequate follow-up, sample size, and blinding. CONCLUSIONS: The use of the gold-standard trial design, the prospective randomized therapeutic study (Level-I or II evidence), does not ensure quality research or reporting. Critical analysis of scientific work is important regardless of the study design. Clinical scientists should be familiar with the CONSORT criteria and adhere to them when reporting clinical trials.
Authors: Michael W Mariscalco; Robert A Magnussen; Divyesh Mehta; Timothy E Hewett; David C Flanigan; Christopher C Kaeding Journal: Am J Sports Med Date: 2013-08-08 Impact factor: 6.202