Joseph J Ruzbarsky1, Sariah Khormaee2, Aaron Daluiski2. 1. Department of Orthopaedics, Hospital for Special Surgery, New York, NY. Electronic address: ruzbarskyj@hss.edu. 2. Department of Orthopaedics, Hospital for Special Surgery, New York, NY.
Abstract
PURPOSE: Randomized controlled trials (RCTs) are the gold standard for comparing clinical interventions. Statistical significance as reported via a P value has been used to determine if a difference between clinical interventions exists in an RCT. However, P values do not clearly convey information about the robustness of a study's conclusions. An emerging metric, called the fragility index (the number of subjects who would need to change outcome category to raise the P value above the .05 threshold), is an indirect measure of how likely a repeat of the trial would reach the same conclusions. This study addressed the fragility of RCTs using dichotomous outcomes in hand surgery. METHODS: Using systematic searching of the MEDLINE database, we identified hand surgery RCTs published in 11 high-impact journals published in the last decade (2007-2017). Studies were identified that involved 2 parallel arms, allocated patients to treatment and control in a 1:1 ratio, and reported statistical significance for a dichotomous variable. The fragility index was calculated using Fisher's exact test, using previously published methods. RESULTS: Five hand surgery RCTs were identified for inclusion reporting a range of fragility indices from 0 to 26. Two of the trials (40%) had a fragility index of 2 or less. Two of the trials (40%) reported that the number of patients lost to follow-up exceeded the fragility index, meaning that results of the patients lost to follow-up could theoretically completely reverse the study conclusions. CONCLUSIONS: The range of fragility indices reported in the recent hand surgery literature is consistent with previous reporting within orthopedic surgery. CLINICAL RELEVANCE: The fragility index is a useful metric to analyze the robustness of the study conclusions that should complement other methods of critical evaluation including the P value or effect sizes. Our results emphasize the need for future efforts to strengthen the robustness of RCT conclusions.
PURPOSE: Randomized controlled trials (RCTs) are the gold standard for comparing clinical interventions. Statistical significance as reported via a P value has been used to determine if a difference between clinical interventions exists in an RCT. However, P values do not clearly convey information about the robustness of a study's conclusions. An emerging metric, called the fragility index (the number of subjects who would need to change outcome category to raise the P value above the .05 threshold), is an indirect measure of how likely a repeat of the trial would reach the same conclusions. This study addressed the fragility of RCTs using dichotomous outcomes in hand surgery. METHODS: Using systematic searching of the MEDLINE database, we identified hand surgery RCTs published in 11 high-impact journals published in the last decade (2007-2017). Studies were identified that involved 2 parallel arms, allocated patients to treatment and control in a 1:1 ratio, and reported statistical significance for a dichotomous variable. The fragility index was calculated using Fisher's exact test, using previously published methods. RESULTS: Five hand surgery RCTs were identified for inclusion reporting a range of fragility indices from 0 to 26. Two of the trials (40%) had a fragility index of 2 or less. Two of the trials (40%) reported that the number of patients lost to follow-up exceeded the fragility index, meaning that results of the patients lost to follow-up could theoretically completely reverse the study conclusions. CONCLUSIONS: The range of fragility indices reported in the recent hand surgery literature is consistent with previous reporting within orthopedic surgery. CLINICAL RELEVANCE: The fragility index is a useful metric to analyze the robustness of the study conclusions that should complement other methods of critical evaluation including the P value or effect sizes. Our results emphasize the need for future efforts to strengthen the robustness of RCT conclusions.
Authors: Lynn Ann Forrester; Kyle L McCormick; Lisa Bonsignore-Opp; Liana J Tedesco; Eric S Baranek; Eugene S Jang; Wakenda K Tyler Journal: J Am Acad Orthop Surg Glob Res Rev Date: 2021-11-19
Authors: Carl L Herndon; Kyle L McCormick; Anastasia Gazgalis; Elise C Bixby; Matthew M Levitsky; Alexander L Neuwirth Journal: Arthroplast Today Date: 2021-10-11
Authors: Nathan P Fackler; Theofilos Karasavvidis; Cooper B Ehlers; Kylie T Callan; Wilson C Lai; Robert L Parisien; Dean Wang Journal: Foot Ankle Int Date: 2022-08-24 Impact factor: 3.569
Authors: Stephen Craig Morris; Anirudh K Gowd; Avinesh Agarwalla; Wesley P Phipatanakul; Nirav H Amin; Joseph N Liu Journal: World J Orthop Date: 2022-09-18