Behnood Bikdeli1,2,3, John W Welsh2, Yasir Akram4, Natdanai Punnanithinont5, Ike Lee6, Nihar R Desai2,7, Sanjay Kaul8, Gregg W Stone1,3, Joseph S Ross2,9,10, Harlan M Krumholz2,7,10,11. 1. Columbia University Medical Center/ New York-Presbyterian Hospital (B.B., G.W.S.). 2. Center for Outcomes Research and Evaluation (CORE) (B.B., J.W.W., N.R.D., J.S.R., H.M.K.), New Haven, CT. 3. Cardiovascular Research Foundation, New York, NY (B.B., G.W.S.). 4. Saint Vincent Hospital, Worcester, MA (Y.A.). 5. Division of Cardiology, Medical College of Georgia, Augusta (N.P.). 6. Yale University School of Medicine (I.L.), New Haven, CT. 7. Section of Cardiovascular Medicine (N.R.D., H.M.K.), New Haven, CT. 8. Division of Cardiology, Cedars-Sinai Medical Center, Los Angeles, CA (S.K.). 9. Section of General Medicine, Department of Internal Medicine (J.S.R.), New Haven, CT. 10. Robert Wood Johnson Foundation Clinical Scholars Program, Department of Internal Medicine (J.S.R., H.M.K.), New Haven, CT. 11. Department of Health Policy and Management, Yale School of Public Health, New Haven, CT (H.M.K.).
Abstract
BACKGROUND: Noninferiority trials are increasingly being performed. However, little is known about their methodological quality. We sought to characterize noninferiority cardiovascular trials published in the highest-impact journals, features that may bias results toward noninferiority, features related to reporting of noninferiority trials, and the time trends. METHODS: We identified cardiovascular noninferiority trials published in JAMA, Lancet, or New England Journal of Medicine from 1990 to 2016. Two independent reviewers extracted the data. Data elements included the noninferiority margin and the success of studies in achieving noninferiority. The proportion of trials showing major or minor features that may have affected the noninferiority inference was determined. Major factors included the lack of presenting the results in both intention-to-treat and per-protocol/as-treated cohorts, α>0.05, the new intervention not being compared with the best alternative, not justifying the noninferiority margin, and exclusion or loss of ≥10% of the cohort. Minor factors included suboptimal blinding, allocation concealment, and others. RESULTS: From 2544 screened studies, we identified 111 noninferiority cardiovascular trials. Noninferiority margins varied widely: risk differences of 0.4% to 25%, hazard ratios of 1.05 to 2.85, odds ratios of 1.1 to 2.0, and relative risks of 1.1 to 1.8. Eighty-six trials claimed noninferiority, of which 20 showed superiority, whereas 23 (21.1%) did not show noninferiority, of which 8 also demonstrated inferiority. Only 7 (6.3%) trials were considered low risk for all the major and minor biasing factors. Among common major factors for bias, 41 (37%) did not confirm the findings in both intention-to-treat and per-protocol/as-treated cohorts and 4 (3.6%) reported discrepant results between intention-to-treat and per-protocol analyses. Forty-three (38.7%) did not justify the noninferiority margin. Overall, 27 (24.3%) underenrolled or had >10% exclusions. Sixty trials (54.0%) were open label. Allocation concealment was not maintained or unclear in 11 (9.9%). Publication of noninferiority trials increased over time (P<0.001). Fifty-two (46.8%) were published after 2010 and had a lower risk of methodological or reporting limitations for major (P=0.03) and minor factors (P=0.002). CONCLUSIONS: Noninferiority trials in highest-impact journals commonly conclude noninferiority of the tested intervention, but vary markedly in the selected noninferiority margin, and frequently have limitations that may impact the inference related to noninferiority.
BACKGROUND: Noninferiority trials are increasingly being performed. However, little is known about their methodological quality. We sought to characterize noninferiority cardiovascular trials published in the highest-impact journals, features that may bias results toward noninferiority, features related to reporting of noninferiority trials, and the time trends. METHODS: We identified cardiovascular noninferiority trials published in JAMA, Lancet, or New England Journal of Medicine from 1990 to 2016. Two independent reviewers extracted the data. Data elements included the noninferiority margin and the success of studies in achieving noninferiority. The proportion of trials showing major or minor features that may have affected the noninferiority inference was determined. Major factors included the lack of presenting the results in both intention-to-treat and per-protocol/as-treated cohorts, α>0.05, the new intervention not being compared with the best alternative, not justifying the noninferiority margin, and exclusion or loss of ≥10% of the cohort. Minor factors included suboptimal blinding, allocation concealment, and others. RESULTS: From 2544 screened studies, we identified 111 noninferiority cardiovascular trials. Noninferiority margins varied widely: risk differences of 0.4% to 25%, hazard ratios of 1.05 to 2.85, odds ratios of 1.1 to 2.0, and relative risks of 1.1 to 1.8. Eighty-six trials claimed noninferiority, of which 20 showed superiority, whereas 23 (21.1%) did not show noninferiority, of which 8 also demonstrated inferiority. Only 7 (6.3%) trials were considered low risk for all the major and minor biasing factors. Among common major factors for bias, 41 (37%) did not confirm the findings in both intention-to-treat and per-protocol/as-treated cohorts and 4 (3.6%) reported discrepant results between intention-to-treat and per-protocol analyses. Forty-three (38.7%) did not justify the noninferiority margin. Overall, 27 (24.3%) underenrolled or had >10% exclusions. Sixty trials (54.0%) were open label. Allocation concealment was not maintained or unclear in 11 (9.9%). Publication of noninferiority trials increased over time (P<0.001). Fifty-two (46.8%) were published after 2010 and had a lower risk of methodological or reporting limitations for major (P=0.03) and minor factors (P=0.002). CONCLUSIONS: Noninferiority trials in highest-impact journals commonly conclude noninferiority of the tested intervention, but vary markedly in the selected noninferiority margin, and frequently have limitations that may impact the inference related to noninferiority.
Authors: Nicolas A Bamat; Osayame A Ekhaguere; Lingqiao Zhang; Dustin D Flannery; Sara C Handley; Heidi M Herrick; Susan S Ellenberg Journal: Clin Trials Date: 2020-07-15 Impact factor: 2.486
Authors: Matthew Dodd; Katherine Fielding; James R Carpenter; Jennifer A Thompson; Diana Elbourne Journal: BMJ Open Date: 2022-01-12 Impact factor: 2.692
Authors: Jaime S Rosa Duque; Xiwei Wang; Daniel Leung; Samuel M S Cheng; Carolyn A Cohen; Xiaofeng Mu; Asmaa Hachim; Yanmei Zhang; Sau Man Chan; Sara Chaothai; Kelvin K H Kwan; Karl C K Chan; John K C Li; Leo L H Luk; Leo C H Tsang; Wilfred H S Wong; Cheuk Hei Cheang; Timothy K Hung; Jennifer H Y Lam; Gilbert T Chua; Winnie W Y Tso; Patrick Ip; Masashi Mori; Niloufar Kavian; Wing Hang Leung; Sophie Valkenburg; Malik Peiris; Wenwei Tu; Yu Lung Lau Journal: Nat Commun Date: 2022-06-28 Impact factor: 17.694
Authors: N Bryce Robinson; Stephen Fremes; Irbaz Hameed; Mohamed Rahouma; Viola Weidenmann; Michelle Demetres; Mahmoud Morsi; Giovanni Soletti; Antonino Di Franco; Marco A Zenati; Shahzad G Raja; David Moher; Faisal Bakaeen; Joanna Chikwe; Deepak L Bhatt; Paul Kurlansky; Leonard N Girardi; Mario Gaudino Journal: JAMA Netw Open Date: 2021-06-01