Jeffrey T Y Chow1,2, Timothy P Turkstra3,2, Edmund Yim2, Philip M Jones4,5,6,7. 1. Department of Epidemiology & Biostatistics, The University of Western Ontario, London, ON, Canada. 2. Schulich School of Medicine & Dentistry, The University of Western Ontario, London, ON, Canada. 3. Department of Anesthesia & Perioperative Medicine, The University of Western Ontario, London, ON, Canada. 4. Department of Epidemiology & Biostatistics, The University of Western Ontario, London, ON, Canada. pjones8@uwo.ca. 5. Department of Anesthesia & Perioperative Medicine, The University of Western Ontario, London, ON, Canada. pjones8@uwo.ca. 6. Schulich School of Medicine & Dentistry, The University of Western Ontario, London, ON, Canada. pjones8@uwo.ca. 7. Rm C3-110 - University Hospital, London Health Sciences Centre, 339 Windermere Rd, London, ON, N6A 5A5, Canada. pjones8@uwo.ca.
Abstract
PURPOSE: Although every randomized clinical trial (RCT) needs participants, determining the ideal number of participants that balances limited resources and the ability to detect a real effect is difficult. Focussing on two-arm, parallel group, superiority RCTs published in six general anesthesiology journals, the objective of this study was to compare the quality of sample size calculations for RCTs published in 2010 vs 2016. METHODS: Each RCT's full text was searched for the presence of a sample size calculation, and the assumptions made by the investigators were compared with the actual values observed in the results. Analyses were only performed for sample size calculations that were amenable to replication, defined as using a clearly identified outcome that was continuous or binary in a standard sample size calculation procedure. RESULTS: The percentage of RCTs reporting all sample size calculation assumptions increased from 51% in 2010 to 84% in 2016. The difference between the values observed in the study and the expected values used for the sample size calculation for most RCTs was usually > 10% of the expected value, with negligible improvement from 2010 to 2016. CONCLUSION: While the reporting of sample size calculations improved from 2010 to 2016, the expected values in these sample size calculations often assumed effect sizes larger than those actually observed in the study. Since overly optimistic assumptions may systematically lead to underpowered RCTs, improvements in how to calculate and report sample sizes in anesthesiology research are needed.
PURPOSE: Although every randomized clinical trial (RCT) needs participants, determining the ideal number of participants that balances limited resources and the ability to detect a real effect is difficult. Focussing on two-arm, parallel group, superiority RCTs published in six general anesthesiology journals, the objective of this study was to compare the quality of sample size calculations for RCTs published in 2010 vs 2016. METHODS: Each RCT's full text was searched for the presence of a sample size calculation, and the assumptions made by the investigators were compared with the actual values observed in the results. Analyses were only performed for sample size calculations that were amenable to replication, defined as using a clearly identified outcome that was continuous or binary in a standard sample size calculation procedure. RESULTS: The percentage of RCTs reporting all sample size calculation assumptions increased from 51% in 2010 to 84% in 2016. The difference between the values observed in the study and the expected values used for the sample size calculation for most RCTs was usually > 10% of the expected value, with negligible improvement from 2010 to 2016. CONCLUSION: While the reporting of sample size calculations improved from 2010 to 2016, the expected values in these sample size calculations often assumed effect sizes larger than those actually observed in the study. Since overly optimistic assumptions may systematically lead to underpowered RCTs, improvements in how to calculate and report sample sizes in anesthesiology research are needed.
Authors: Julio Meza; Rilwan Babajide; Ragheed Saoud; Jeanne M Horowitz; David D Casalino; Adam B Murphy; Jamila Sweis; Josephine Abelleira; Irene Helenowski; Borko Jovanovic; Scott Eggener; Frank H Miller Journal: BMC Urol Date: 2022-07-18 Impact factor: 2.090