David M Kent1, Jason Nelson1, Issa J Dahabreh1,2,3,4, Peter M Rothwell5, Douglas G Altman6, Rodney A Hayward7. 1. Predictive Analytics and Comparative Effectiveness Center, Tufts Medical Center/Tufts University School of Medicine, Boston, MA, USA. 2. Center for Evidence-Based Medicine. 3. Department of Health Services, Policy & Practice. 4. Department of Epidemiology, Brown University, Providence, RI, USA. 5. Stroke Prevention Research Unit, Nuffield Department of Clinical Neuroscience, University of Oxford, Oxford, UK. 6. Centre for Statistics in Medicine, Nuffield Department of Orthopaedics, Rheumatology & Musculoskeletal Sciences, University of Oxford, Oxford, UK. 7. Department of Internal Medicine and Department of Health Management and Policy, University of Michigan, Ann Arbor, MI, USA.
Abstract
Background: Risk of the outcome is a mathematical determinant of the absolute treatment benefit of an intervention, yet this can vary substantially within a trial population, complicating the interpretation of trial results. Methods: We developed risk models using Cox or logistic regression on a set of large publicly available randomized controlled trials (RCTs). We evaluated risk heterogeneity using the extreme quartile risk ratio (EQRR, the ratio of outcome rates in the lowest risk quartile to that in the highest) and skewness using the median to mean risk ratio (MMRR, the ratio of risk in the median risk patient to the average). We also examined heterogeneity of treatment effects (HTE) across risk strata. Results: We describe 39 analyses using data from 32 large trials, with event rates across studies ranging from 3% to 63% (median = 15%, 25th-75th percentile = 9-29%). C-statistics of risk models ranged from 0.59 to 0.89 (median = 0.70, 25th-75th percentile = 0.65-0.71). The EQRR ranged from 1.8 to 50.7 (median = 4.3, 25th-75th percentile = 3.0-6.1). The MMRR ranged from 0.4 to 1.0 (median = 0.86, 25th-75th percentile = 0.80-0.92). EQRRs were predictably higher and MMRRs predictably lower as the c-statistic increased or the overall outcome incidence decreased. Among 18 comparisons with a significant overall treatment effect, there was a significant interaction between treatment and baseline risk on the proportional scale in only one. The difference in the absolute risk reduction between extreme risk quartiles ranged from -3.2 to 28.3% (median = 5.1%; 25th-75th percentile = 0.3-10.9). Conclusions: There is typically substantial variation in outcome risk in clinical trials, commonly leading to clinically significant differences in absolute treatment effects Most patients have outcome risks lower than the trial average reflected in the summary result. Risk-stratified trial analyses are feasible and may be clinically informative, particularly when the outcome is predictable and uncommon.
Background: Risk of the outcome is a mathematical determinant of the absolute treatment benefit of an intervention, yet this can vary substantially within a trial population, complicating the interpretation of trial results. Methods: We developed risk models using Cox or logistic regression on a set of large publicly available randomized controlled trials (RCTs). We evaluated risk heterogeneity using the extreme quartile risk ratio (EQRR, the ratio of outcome rates in the lowest risk quartile to that in the highest) and skewness using the median to mean risk ratio (MMRR, the ratio of risk in the median risk patient to the average). We also examined heterogeneity of treatment effects (HTE) across risk strata. Results: We describe 39 analyses using data from 32 large trials, with event rates across studies ranging from 3% to 63% (median = 15%, 25th-75th percentile = 9-29%). C-statistics of risk models ranged from 0.59 to 0.89 (median = 0.70, 25th-75th percentile = 0.65-0.71). The EQRR ranged from 1.8 to 50.7 (median = 4.3, 25th-75th percentile = 3.0-6.1). The MMRR ranged from 0.4 to 1.0 (median = 0.86, 25th-75th percentile = 0.80-0.92). EQRRs were predictably higher and MMRRs predictably lower as the c-statistic increased or the overall outcome incidence decreased. Among 18 comparisons with a significant overall treatment effect, there was a significant interaction between treatment and baseline risk on the proportional scale in only one. The difference in the absolute risk reduction between extreme risk quartiles ranged from -3.2 to 28.3% (median = 5.1%; 25th-75th percentile = 0.3-10.9). Conclusions: There is typically substantial variation in outcome risk in clinical trials, commonly leading to clinically significant differences in absolute treatment effects Most patients have outcome risks lower than the trial average reflected in the summary result. Risk-stratified trial analyses are feasible and may be clinically informative, particularly when the outcome is predictable and uncommon.
Authors: Eric J Eichhorn; Michael J Domanski; Heidi Krause-Steinrauf; Michael R Bristow; Philip W Lavori Journal: N Engl J Med Date: 2001-05-31 Impact factor: 91.245
Authors: Sara T Brookes; Elise Whitely; Matthias Egger; George Davey Smith; Paul A Mulheran; Tim J Peters Journal: J Clin Epidemiol Date: 2004-03 Impact factor: 6.437
Authors: C P Cannon; W S Weintraub; L A Demopoulos; R Vicari; M J Frey; N Lakkis; F J Neumann; D H Robertson; P T DeLucca; P M DiBattiste; C M Gibson; E Braunwald Journal: N Engl J Med Date: 2001-06-21 Impact factor: 91.245
Authors: David M Kent; Rodney A Hayward; John L Griffith; Sandeep Vijan; Joni R Beshansky; Robert M Califf; Harry P Selker Journal: Am J Med Date: 2002-08-01 Impact factor: 4.965
Authors: Anil K Chaturvedi; Hormuzd A Katki; Stephanie A Kovalchik; Martin Tammemagi; Christine D Berg; Neil E Caporaso; Tom L Riley; Mary Korch; Gerard A Silvestri Journal: N Engl J Med Date: 2013-07-18 Impact factor: 91.245
Authors: Johannes A N Dorresteijn; Frank L J Visseren; Paul M Ridker; Annemarie M J Wassink; Nina P Paynter; Ewout W Steyerberg; Yolanda van der Graaf; Nancy R Cook Journal: BMJ Date: 2011-10-03
Authors: Susanna F Fustolo-Gunnink; Karin Fijnvandraat; David van Klaveren; Simon J Stanworth; Anna Curley; Wes Onland; Ewout W Steyerberg; Ellen de Kort; Esther J d'Haens; Christian V Hulzebos; Elise J Huisman; Willem P de Boode; Enrico Lopriore; Johanna G van der Bom Journal: Blood Date: 2019-12-26 Impact factor: 22.113
Authors: Muthiah Vaduganathan; Amulya Nagarur; Arman Qamar; Ravi B Patel; Ann Marie Navar; Eric D Peterson; Deepak L Bhatt; Gregg C Fonarow; Clyde W Yancy; Javed Butler Journal: Circulation Date: 2017-11-13 Impact factor: 29.690
Authors: Natalia Olchanski; David van Klaveren; Joshua T Cohen; John B Wong; Robin Ruthazer; David M Kent Journal: Acta Diabetol Date: 2021-01-30 Impact factor: 4.280
Authors: Kathryn M Stadeli; Mariam N Hantouli; Elena G Brewer; Elizabeth Austin; Kemi M Doll; Danielle C Lavallee; Giana H Davidson Journal: Am J Surg Date: 2019-07-29 Impact factor: 2.565