Austin L Johnson1, Sheridan Evans2, Jake X Checketts2, Jared T Scott3, Cole Wayant2, Mark Johnson4, Brent Norris5, Matt Vassar3. 1. Oklahoma State University Center for Health Sciences, Tulsa, OK, United States. Electronic address: austin.johnson14@okstate.edu. 2. Oklahoma State University Center for Health Sciences, Tulsa, OK, United States. 3. Oklahoma State University Center for Health Sciences, Tulsa, OK, United States; Oklahoma State University Medical Center - Department of Orthopaedics, Tulsa, OK, United States. 4. Oklahoma State University Medical Center - Department of Orthopaedics, Tulsa, OK, United States. 5. Orthopaedic & Trauma Services of Oklahoma, Tulsa, OK, United States; Oklahoma State University Medical Center - Department of Orthopaedics, Tulsa, OK, United States.
Abstract
INTRODUCTION: A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. P values less than .05 but greater than .005 would be reclassified as "suggestive", whereas P values less than .005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. METHODS: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. RESULTS: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than .05 and 60.5% (75/124) had a P value greater than .05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than .005, while 49.0% (24/49) would be reclassified as suggestive. CONCLUSION: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.
INTRODUCTION: A recent proposal suggests changing the threshold for statistical significance from a P value of .05 to .005 to minimize bias and increase reproducibility of future studies. P values less than .05 but greater than .005 would be reclassified as "suggestive", whereas P values less than .005 would be considered significant. The present study explores how lowering the P value threshold would affect the interpretation of previously published orthopaedic trauma randomized controlled trials (RCTs) and whether outcomes from these trials would maintain statistical significance under the proposed P value threshold. METHODS: All RCTs published between January 01, 2016 and January 31, 2018 in the Journal of Orthopaedic Trauma, Injury, and Archives of Orthopaedic and Trauma Surgery were screened by at least 2 authors. Data from included trials were extracted in blinded and duplicate fashion. All P values for primary endpoints were included from each study. RESULTS: We identified 124 primary endpoints from 48 trials: 39.5% (49/124) of endpoints had a P value less than .05 and 60.5% (75/124) had a P value greater than .05. Overall, 51.0% (25/49) of statistically significant primary endpoints were less than .005, while 49.0% (24/49) would be reclassified as suggestive. CONCLUSION: Based on our results, adopting a lower threshold of significance would heavily alter the significance of orthopaedic trauma RCTs and should be further evaluated and cautiously considered when viewing the effect such a proposal on orthopaedic practice.
Authors: Ann M Bruno; Ashley E Shea; Brett D Einerson; Torri D Metz; Amanda A Allshouse; James R Scott; Nathan R Blue Journal: Am J Perinatol Date: 2021-06-24 Impact factor: 3.079