Leath Abdullah1, Daniel E Davis2, Peter D Fabricant3, Keith Baldwin3, Surena Namdari4. 1. Drexel University College of Medicine, 2900 West Queen Lane, Philadelphia, PA 19129. 2. Department of Orthopaedic Surgery, Thomas Jefferson University Hospital, 1025 Walnut Street, Suite 516, Philadelphia, PA 19107. 3. Department of Orthopaedic Surgery, Children's Hospital of Philadelphia, 2 Wood Building, Philadelphia, PA 19104. 4. The Rothman Institute at Thomas Jefferson University, 925 Chestnut St, 5th Floor, Philadelphia, PA 19107. E-mail address: Surena.Namdari@rothmaninstitute.com.
Abstract
BACKGROUND: Randomized controlled trials (RCTs) are considered the gold standard in evidence-based medicine. Underpowered RCTs that describe comparative outcomes without significance are of questionable benefit. The present study hypothesizes that a substantial proportion of RCTs in the orthopaedic literature that do not note significant differences between groups are inadequately powered. METHODS: Using the ISI Web of Science database, we searched all English-language journals in the orthopaedic category for RCTs published from January 2012 to December 2013. Qualifying articles were analyzed with regard to whether the null hypothesis was rejected (a positive study) for the primary outcome or if it was not (a negative study), whether a power analysis was performed, and whether the study was adequately powered. We performed a power analysis based on the primary outcome or outcomes of interest for the studies that did not describe a power analysis. RESULTS: After inclusion and exclusion criteria were applied, 456 RCTs were selected for complete review. Of those studies, 215 (47.1%) had negative findings and 241 (52.9%) had positive findings for primary outcomes. Twenty-five studies that failed to reject the null hypothesis noted inadequate power in the study. On the basis of our own power analyses, we found an additional thirty-five negative studies without power calculations to be underpowered. Sixty (27.9%) of the 215 negative studies were underpowered. Following binary logistic regression, only the journal impact factor was a significant predictor of whether a study was underpowered. CONCLUSIONS: If an RCT lacks adequate statistical power to identify a clinically meaningful absence of a difference between groups, there is an unacceptable risk of inappropriately failing to reject the null hypothesis. The present study found that a sizable proportion of RCTs in orthopaedic surgery in which the null hypothesis is rejected are inadequately powered. Researchers should consider this when designing clinical trials, and journal editors and reviewers should be wary of underpowered RCTs when considering manuscripts for publication.
BACKGROUND: Randomized controlled trials (RCTs) are considered the gold standard in evidence-based medicine. Underpowered RCTs that describe comparative outcomes without significance are of questionable benefit. The present study hypothesizes that a substantial proportion of RCTs in the orthopaedic literature that do not note significant differences between groups are inadequately powered. METHODS: Using the ISI Web of Science database, we searched all English-language journals in the orthopaedic category for RCTs published from January 2012 to December 2013. Qualifying articles were analyzed with regard to whether the null hypothesis was rejected (a positive study) for the primary outcome or if it was not (a negative study), whether a power analysis was performed, and whether the study was adequately powered. We performed a power analysis based on the primary outcome or outcomes of interest for the studies that did not describe a power analysis. RESULTS: After inclusion and exclusion criteria were applied, 456 RCTs were selected for complete review. Of those studies, 215 (47.1%) had negative findings and 241 (52.9%) had positive findings for primary outcomes. Twenty-five studies that failed to reject the null hypothesis noted inadequate power in the study. On the basis of our own power analyses, we found an additional thirty-five negative studies without power calculations to be underpowered. Sixty (27.9%) of the 215 negative studies were underpowered. Following binary logistic regression, only the journal impact factor was a significant predictor of whether a study was underpowered. CONCLUSIONS: If an RCT lacks adequate statistical power to identify a clinically meaningful absence of a difference between groups, there is an unacceptable risk of inappropriately failing to reject the null hypothesis. The present study found that a sizable proportion of RCTs in orthopaedic surgery in which the null hypothesis is rejected are inadequately powered. Researchers should consider this when designing clinical trials, and journal editors and reviewers should be wary of underpowered RCTs when considering manuscripts for publication.
Authors: Devin R Mangold; Eric R Wagner; Robert H Cofield; Joaquin Sanchez-Sotelo; John W Sperling Journal: Int Orthop Date: 2019-07-26 Impact factor: 3.075
Authors: Armin Arshi; Peter D Fabricant; Derek E Go; Riley J Williams; David R McAllister; Kristofer J Jones Journal: Cartilage Date: 2017-12-15 Impact factor: 4.634
Authors: Stephen Craig Morris; Anirudh K Gowd; Avinesh Agarwalla; Wesley P Phipatanakul; Nirav H Amin; Joseph N Liu Journal: World J Orthop Date: 2022-09-18