OBJECTIVE: Randomized controlled trials (RCTs) are considered the highest level of evidence to inform clinical practice. However, the reproducibility crisis has raised concerns about the scientific rigor of published RCT findings. Some advocate for a lower p-value threshold. We aimed to review published OB/Gyn topical RCTs in three representative OB/Gyn journals and three high impact non-OB/Gyn journals to determine if their interpretations would change with adoption of a p-value threshold for significance of 0.005. Secondarily, we evaluated if there were differences in methodologic characteristics between those that did and did not lose significance. STUDY DESIGN: A manual search was performed to identify all OB/Gyn RCTs published in the selected journals between July 2017 and June 2019. Data were collected on primary outcome(s), methodology, and p-values. We determined the proportion of primary outcomes that would remain statistically significant with adoption of a p-value significance threshold of 0.005 versus be reinterpreted as "suggestive" (defined as p-value between 0.005 and 0.05). Chi-square or Fisher's exact test were used to compare study characteristics. RESULTS: Overall, 202 RCTs met inclusion criteria; 52% in obstetrics and 48% in gynecology. Of 90 studies considered significant with p <0.05 at the time of publication, 54.4% (n = 49) would maintain significant (p < 0.005), while 45.6% (n = 41) would become suggestive using the lower threshold. Most RCTs utilized a single (90.1%) versus composite (8.9%) primary outcome type, used an intent-to-treat analysis (73.3%), and studied a drug intervention (46.5%). Methodologically, 23.7% did not prespecify analysis type, 28.2% did not meet the pre-determined sample size, and 9.4% did not report an a priori sample size calculation. Studies maintaining significance were more likely to be international and report a funding source. CONCLUSION: Adopting a p-value significance threshold of 0.005 would require reinterpretation of almost half of RCT results in the OB/Gyn literature. Highly variable methodological quality was identified. KEY POINTS: · New p-value threshold results in reinterpretation of nearly half of RCT results in OB/Gyn literature.. · Highly variable methodological quality was identified.. · Reduced use of binary interpretations of significance is necessary.. Thieme. All rights reserved.
OBJECTIVE: Randomized controlled trials (RCTs) are considered the highest level of evidence to inform clinical practice. However, the reproducibility crisis has raised concerns about the scientific rigor of published RCT findings. Some advocate for a lower p-value threshold. We aimed to review published OB/Gyn topical RCTs in three representative OB/Gyn journals and three high impact non-OB/Gyn journals to determine if their interpretations would change with adoption of a p-value threshold for significance of 0.005. Secondarily, we evaluated if there were differences in methodologic characteristics between those that did and did not lose significance. STUDY DESIGN: A manual search was performed to identify all OB/Gyn RCTs published in the selected journals between July 2017 and June 2019. Data were collected on primary outcome(s), methodology, and p-values. We determined the proportion of primary outcomes that would remain statistically significant with adoption of a p-value significance threshold of 0.005 versus be reinterpreted as "suggestive" (defined as p-value between 0.005 and 0.05). Chi-square or Fisher's exact test were used to compare study characteristics. RESULTS: Overall, 202 RCTs met inclusion criteria; 52% in obstetrics and 48% in gynecology. Of 90 studies considered significant with p <0.05 at the time of publication, 54.4% (n = 49) would maintain significant (p < 0.005), while 45.6% (n = 41) would become suggestive using the lower threshold. Most RCTs utilized a single (90.1%) versus composite (8.9%) primary outcome type, used an intent-to-treat analysis (73.3%), and studied a drug intervention (46.5%). Methodologically, 23.7% did not prespecify analysis type, 28.2% did not meet the pre-determined sample size, and 9.4% did not report an a priori sample size calculation. Studies maintaining significance were more likely to be international and report a funding source. CONCLUSION: Adopting a p-value significance threshold of 0.005 would require reinterpretation of almost half of RCT results in the OB/Gyn literature. Highly variable methodological quality was identified. KEY POINTS: · New p-value threshold results in reinterpretation of nearly half of RCT results in OB/Gyn literature.. · Highly variable methodological quality was identified.. · Reduced use of binary interpretations of significance is necessary.. Thieme. All rights reserved.
Authors: Austin L Johnson; Sheridan Evans; Jake X Checketts; Jared T Scott; Cole Wayant; Mark Johnson; Brent Norris; Matt Vassar Journal: Injury Date: 2019-08-12 Impact factor: 2.586
Authors: Daniel J Benjamin; James O Berger; Magnus Johannesson; Brian A Nosek; E-J Wagenmakers; Richard Berk; Kenneth A Bollen; Björn Brembs; Lawrence Brown; Colin Camerer; David Cesarini; Christopher D Chambers; Merlise Clyde; Thomas D Cook; Paul De Boeck; Zoltan Dienes; Anna Dreber; Kenny Easwaran; Charles Efferson; Ernst Fehr; Fiona Fidler; Andy P Field; Malcolm Forster; Edward I George; Richard Gonzalez; Steven Goodman; Edwin Green; Donald P Green; Anthony G Greenwald; Jarrod D Hadfield; Larry V Hedges; Leonhard Held; Teck Hua Ho; Herbert Hoijtink; Daniel J Hruschka; Kosuke Imai; Guido Imbens; John P A Ioannidis; Minjeong Jeon; James Holland Jones; Michael Kirchler; David Laibson; John List; Roderick Little; Arthur Lupia; Edouard Machery; Scott E Maxwell; Michael McCarthy; Don A Moore; Stephen L Morgan; Marcus Munafó; Shinichi Nakagawa; Brendan Nyhan; Timothy H Parker; Luis Pericchi; Marco Perugini; Jeff Rouder; Judith Rousseau; Victoria Savalei; Felix D Schönbrodt; Thomas Sellke; Betsy Sinclair; Dustin Tingley; Trisha Van Zandt; Simine Vazire; Duncan J Watts; Christopher Winship; Robert L Wolpert; Yu Xie; Cristobal Young; Jonathan Zinman; Valen E Johnson Journal: Nat Hum Behav Date: 2018-01
Authors: William A Grobman; Madeline M Rice; Uma M Reddy; Alan T N Tita; Robert M Silver; Gail Mallett; Kim Hill; Elizabeth A Thom; Yasser Y El-Sayed; Annette Perez-Delboy; Dwight J Rouse; George R Saade; Kim A Boggess; Suneet P Chauhan; Jay D Iams; Edward K Chien; Brian M Casey; Ronald S Gibbs; Sindhu K Srinivas; Geeta K Swamy; Hyagriv N Simhan; George A Macones Journal: N Engl J Med Date: 2018-08-09 Impact factor: 91.245