Lauri Raittio1, Antti Launonen2, Ville M Mattila3,2, Aleksi Reito2. 1. The Faculty of Medicine and Health Technology, Tampere University, Arvo Ylpön katu 34, 33520, Tampere, Finland. lauri.raittio@tuni.fi. 2. Department of Orthopaedics and Traumatology, Tampere University Hospital, Teiskontie 35, 33520, Tampere, Finland. 3. The Faculty of Medicine and Health Technology, Tampere University, Arvo Ylpön katu 34, 33520, Tampere, Finland.
Abstract
BACKGROUND: Randomized controlled trials in orthopaedics are powered to mainly find large effect sizes. A possible discrepancy between the estimated and the real mean difference is a challenge for statistical inference based on p-values. We explored the justifications of the mean difference estimates used in power calculations. The assessment of distribution of observations in the primary outcome and the possibility of ceiling effects were also assessed. METHODS: Systematic review of the randomized controlled trials with power calculations in eight clinical orthopaedic journals published between 2016 and 2019. Trials with one continuous primary outcome and 1:1 allocation were eligible. Rationales and references for the mean difference estimate were recorded from the Methods sections. The possibility of ceiling effect was addressed by the assessment of the weighted mean and standard deviation of the primary outcome and its elaboration in the Discussion section of each RCT where available. RESULTS: 264 trials were included in this study. Of these, 108 (41 %) trials provided some rationale or reference for the mean difference estimate. The most common rationales or references for the estimate of mean difference were minimal clinical important difference (16 %), observational studies on the same subject (8 %) and the 'clinical relevance' of the authors (6 %). In a third of the trials, the weighted mean plus 1 standard deviation of the primary outcome reached over the best value in the patient-reported outcome measure scale, indicating the possibility of ceiling effect in the outcome. CONCLUSIONS: The chosen mean difference estimates in power calculations are rarely properly justified in orthopaedic trials. In general, trials with a patient-reported outcome measure as the primary outcome do not assess or report the possibility of the ceiling effect in the primary outcome or elaborate further in the Discussion section.
BACKGROUND: Randomized controlled trials in orthopaedics are powered to mainly find large effect sizes. A possible discrepancy between the estimated and the real mean difference is a challenge for statistical inference based on p-values. We explored the justifications of the mean difference estimates used in power calculations. The assessment of distribution of observations in the primary outcome and the possibility of ceiling effects were also assessed. METHODS: Systematic review of the randomized controlled trials with power calculations in eight clinical orthopaedic journals published between 2016 and 2019. Trials with one continuous primary outcome and 1:1 allocation were eligible. Rationales and references for the mean difference estimate were recorded from the Methods sections. The possibility of ceiling effect was addressed by the assessment of the weighted mean and standard deviation of the primary outcome and its elaboration in the Discussion section of each RCT where available. RESULTS: 264 trials were included in this study. Of these, 108 (41 %) trials provided some rationale or reference for the mean difference estimate. The most common rationales or references for the estimate of mean difference were minimal clinical important difference (16 %), observational studies on the same subject (8 %) and the 'clinical relevance' of the authors (6 %). In a third of the trials, the weighted mean plus 1 standard deviation of the primary outcome reached over the best value in the patient-reported outcome measure scale, indicating the possibility of ceiling effect in the outcome. CONCLUSIONS: The chosen mean difference estimates in power calculations are rarely properly justified in orthopaedic trials. In general, trials with a patient-reported outcome measure as the primary outcome do not assess or report the possibility of the ceiling effect in the primary outcome or elaborate further in the Discussion section.
Authors: Raymond W J G Ostelo; Rick A Deyo; P Stratford; Gordon Waddell; Peter Croft; Michael Von Korff; Lex M Bouter; Henrica C de Vet Journal: Spine (Phila Pa 1976) Date: 2008-01-01 Impact factor: 3.468
Authors: Tahira Devji; Gordon H Guyatt; Lyubov Lytvyn; Romina Brignardello-Petersen; Farid Foroutan; Behnam Sadeghirad; Rachelle Buchbinder; Rudolf W Poolman; Ian A Harris; Alonso Carrasco-Labra; Reed A C Siemieniuk; Per O Vandvik Journal: BMJ Open Date: 2017-05-11 Impact factor: 2.692
Authors: Cecilia A C Prinsen; Sunita Vohra; Michael R Rose; Maarten Boers; Peter Tugwell; Mike Clarke; Paula R Williamson; Caroline B Terwee Journal: Trials Date: 2016-09-13 Impact factor: 2.279