X Huang1, B Chen2, L Thabane3, J D Adachi4, G Li5,6. 1. Center for Clinical Epidemiology and Methodology (CCEM), Guangdong Second Provincial General Hospital, Guangzhou, China. 2. Department of Endocrinology, Guangdong Second Provincial General Hospital, Guangzhou, China. 3. Department of Health research methods, Evidence, and Impact (HEI), McMaster University, 1280 Main St West, Hamilton, ON, L8S 4 L8, Canada. 4. Department of Medicine, McMaster University, Hamilton, ON, Canada. 5. Center for Clinical Epidemiology and Methodology (CCEM), Guangdong Second Provincial General Hospital, Guangzhou, China. lig28@mcmaster.ca. 6. Department of Health research methods, Evidence, and Impact (HEI), McMaster University, 1280 Main St West, Hamilton, ON, L8S 4 L8, Canada. lig28@mcmaster.ca.
Abstract
This is the first report on the fragility of results from randomized controlled trials (RCTs) for the treatment of osteoporosis. The results of aforementioned RCTs appear to depend on a small number of events and are generally statistically fragile. INTRODUCTION: Osteoporosis remains a health concern worldwide. Evidence-based guideline recommendations that are mainly based on results of clinical trials are important to clinical decision-making. The fragility index (FI) is a novel statistical metric to measure the fragility of results from an RCT. Our study aimed to analyze the fragility of the clinical trials referenced in the guidelines for the treatment of osteoporosis. METHODS: Trials were included if they investigated primary osteoporosis, randomized patients to treatment or control in a 1:1 design, and reported fracture outcome as the primary endpoint. The FI and fragility quotient (FQ) were calculated for assessing the robustness of results from the eligible RCTs. An FI was defined as the minimum number of events in the intervention group that needs to change from a non-event to an event in order to render a significant result non-significant (or vice versa). The FQ was calculated by dividing the FI by the sample size of the trial. RESULTS: Of the 372 RCTs identified from the guidelines, 42 were eligible for analyses. Their median FI was 10 (25th-75th percentile [Q1-Q3]: 4-18), with a median FQ of 0.007 (Q1-Q3: 0.0017-0.019). Approximately one third of the RCTs had a FI of less than or equal to 5. There were 17 (40.5%) trials where the number of patients lost to follow-up was greater than the FI. The FI was significantly associated with sample size, journal impact factor, and the percent of patients lost to follow-up. CONCLUSION: Results from some RCTs supporting guideline recommendations for the treatment of osteoporosis depend on a small number of events. The FI and FQ may provide additional, intuitive metrics to help interpret the robustness of trial results.
This is the first report on the fragility of results from randomized controlled trials (RCTs) for the treatment of osteoporosis. The results of aforementioned RCTs appear to depend on a small number of events and are generally statistically fragile. INTRODUCTION:Osteoporosis remains a health concern worldwide. Evidence-based guideline recommendations that are mainly based on results of clinical trials are important to clinical decision-making. The fragility index (FI) is a novel statistical metric to measure the fragility of results from an RCT. Our study aimed to analyze the fragility of the clinical trials referenced in the guidelines for the treatment of osteoporosis. METHODS: Trials were included if they investigated primary osteoporosis, randomized patients to treatment or control in a 1:1 design, and reported fracture outcome as the primary endpoint. The FI and fragility quotient (FQ) were calculated for assessing the robustness of results from the eligible RCTs. An FI was defined as the minimum number of events in the intervention group that needs to change from a non-event to an event in order to render a significant result non-significant (or vice versa). The FQ was calculated by dividing the FI by the sample size of the trial. RESULTS: Of the 372 RCTs identified from the guidelines, 42 were eligible for analyses. Their median FI was 10 (25th-75th percentile [Q1-Q3]: 4-18), with a median FQ of 0.007 (Q1-Q3: 0.0017-0.019). Approximately one third of the RCTs had a FI of less than or equal to 5. There were 17 (40.5%) trials where the number of patients lost to follow-up was greater than the FI. The FI was significantly associated with sample size, journal impact factor, and the percent of patients lost to follow-up. CONCLUSION: Results from some RCTs supporting guideline recommendations for the treatment of osteoporosis depend on a small number of events. The FI and FQ may provide additional, intuitive metrics to help interpret the robustness of trial results.
Authors: Michael Walsh; Sadeesh K Srinathan; Daniel F McAuley; Marko Mrkobrada; Oren Levine; Christine Ribic; Amber O Molnar; Neil D Dattani; Andrew Burke; Gordon Guyatt; Lehana Thabane; Stephen D Walter; Janice Pogue; P J Devereaux Journal: J Clin Epidemiol Date: 2014-02-05 Impact factor: 6.437
Authors: Kieran F Docherty; Ross T Campbell; Pardeep S Jhund; Mark C Petrie; John J V McMurray Journal: Eur Heart J Date: 2017-02-01 Impact factor: 29.983
Authors: Nathan P Fackler; Cooper B Ehlers; Kylie T Callan; Arya Amirhekmat; Eric J Smith; Robert L Parisien; Dean Wang Journal: Orthop J Sports Med Date: 2022-05-10
Authors: Benjamin R Baer; Stephen E Fremes; Mario Gaudino; Mary Charlson; Martin T Wells Journal: BMC Med Res Methodol Date: 2021-11-20 Impact factor: 4.615
Authors: Nathan P Fackler; Theofilos Karasavvidis; Cooper B Ehlers; Kylie T Callan; Wilson C Lai; Robert L Parisien; Dean Wang Journal: Foot Ankle Int Date: 2022-08-24 Impact factor: 3.569