Seth A Berkowitz1,2, Kara E Rudolph3, Sanjay Basu4,5,6,7,8. 1. Division of General Medicine and Clinical Epidemiology, Department of Medicine, University of North Carolina School of Medicine, Chapel Hill (S.A.B.). 2. Cecil G. Sheps Center for Health Services Research, University of North Carolina at Chapel Hill (S.A.B.). 3. Department of Emergency Medicine, School of Medicine, University of California, Davis, Sacramento (K.E.R.). 4. Center for Primary Care and Outcomes Research (S.B.), Stanford University, CA. 5. Center for Population Health Sciences (S.B.), Stanford University, CA. 6. Department of Medicine (S.B.), Stanford University, CA. 7. Department of Health Research and Policy (S.B.), Stanford University, CA. 8. Center for Primary Care, Harvard Medical School, Boston, MA (S.B.).
Abstract
BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.
BACKGROUND: Recent multisite trials reveal striking heterogeneities in results between trial sites. These may be because of population differences indicating different treatment benefits among different types of participants or site anomalies, such as failures to adhere to study protocols that could negatively affect study validity. We sought to determine whether a new data analysis strategy-transportability methods-could suggest site anomalies not readily identified through standard methods. METHODS AND RESULTS: We applied transportability methods to 2 large, multicenter cardiovascular disease treatment trials: the TOPCAT trial (Treatment of Preserved Cardiac Function Heart Failure With an Aldosterone Antagonist; n=3445) comparing spironolactone to placebo for heart failure (for which site anomalies were suspected) and the ACCORD BP trial (Action to Control Cardiovascular Risk in Diabetes-Blood Pressure; n=4733) comparing intensive-to-standard blood pressure treatment (for which site anomalies were not suspected). The transportability methods give expected results by standardizing from one site to another using data on participant covariates. The difference between the expected and observed results was assessed using calibration tests to identify whether treatment-effect differences between sites could be explained by participant population characteristics. Standard regression methods did not detect heterogeneities in TOPCAT between Russia/Georgia study sites suspected of study protocol violations and sites in the Americas ( P=0.12 for difference in primary cardiovascular outcome; P=0.20 for difference in total mortality). The transportability methods, however, detected the difference between Russia/Georgia sites and sites in the Americas ( P<0.001) and found that measured participant characteristics did not explain the between-site discrepancies. The transport methods found no such discrepancies between sites in ACCORD BP, suggesting participant characteristics explained between-site differences. CONCLUSIONS: Transportability methods may be superior to standard approaches for detecting anomalies within multicenter randomized trials and assist data monitoring boards to determine whether important treatment-effect heterogeneities can be attributed to participant differences or potentially to site performance differences requiring further investigation.
Authors: Daniel Westreich; Jessie K Edwards; Catherine R Lesko; Elizabeth Stuart; Stephen R Cole Journal: Am J Epidemiol Date: 2017-10-15 Impact factor: 4.897
Authors: Simon de Denus; Eileen O'Meara; Akshay S Desai; Brian Claggett; Eldrin F Lewis; Grégoire Leclair; Martin Jutras; Joël Lavoie; Scott D Solomon; Bertram Pitt; Marc A Pfeffer; Jean L Rouleau Journal: N Engl J Med Date: 2017-04-27 Impact factor: 91.245
Authors: Marc A Pfeffer; Brian Claggett; Susan F Assmann; Robin Boineau; Inder S Anand; Nadine Clausell; Akshay S Desai; Rafael Diaz; Jerome L Fleg; Ivan Gordeev; John F Heitner; Eldrin F Lewis; Eileen O'Meara; Jean-Lucien Rouleau; Jeffrey L Probstfield; Tamaz Shaburishvili; Sanjiv J Shah; Scott D Solomon; Nancy K Sweitzer; Sonja M McKinlay; Bertram Pitt Journal: Circulation Date: 2014-11-18 Impact factor: 29.690
Authors: Bertram Pitt; Marc A Pfeffer; Susan F Assmann; Robin Boineau; Inder S Anand; Brian Claggett; Nadine Clausell; Akshay S Desai; Rafael Diaz; Jerome L Fleg; Ivan Gordeev; Brian Harty; John F Heitner; Christopher T Kenwood; Eldrin F Lewis; Eileen O'Meara; Jeffrey L Probstfield; Tamaz Shaburishvili; Sanjiv J Shah; Scott D Solomon; Nancy K Sweitzer; Song Yang; Sonja M McKinlay Journal: N Engl J Med Date: 2014-04-10 Impact factor: 91.245
Authors: William C Cushman; Gregory W Evans; Robert P Byington; David C Goff; Richard H Grimm; Jeffrey A Cutler; Denise G Simons-Morton; Jan N Basile; Marshall A Corson; Jeffrey L Probstfield; Lois Katz; Kevin A Peterson; William T Friedewald; John B Buse; J Thomas Bigger; Hertzel C Gerstein; Faramarz Ismail-Beigi Journal: N Engl J Med Date: 2010-03-14 Impact factor: 91.245
Authors: Kara E Rudolph; Nicole M Schmidt; M Maria Glymour; Rebecca Crowder; Jessica Galin; Jennifer Ahern; Theresa L Osypuk Journal: Epidemiology Date: 2018-03 Impact factor: 4.822