Shanil Ebrahim1, Zahra N Sohani2, Luis Montoya3, Arnav Agarwal4, Kristian Thorlund5, Edward J Mills6, John P A Ioannidis7. 1. Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada3Department of Anesthesia, McMaster University4Depa. 2. Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada5Population Genomics Program, McMaster University. 3. University Health Network, Toronto, Ontario, Canada. 4. Faculty of Health Sciences, McMaster University. 5. Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada. 6. Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California2Department of Clinical Epidemiology and Biostatistics, McMaster University, Hamilton, Ontario, Canada8Faculty of Health Sciences, University of Ottawa. 7. Stanford Prevention Research Center, Department of Medicine, Stanford University, Stanford, California9Department of Health Research and Policy, Stanford University School of Medicine10Department of Statistics, Stanford University School of Humanities and.
Abstract
IMPORTANCE: Reanalyses of randomized clinical trial (RCT) data may help the scientific community assess the validity of reported trial results. OBJECTIVES: To identify published reanalyses of RCT data, to characterize methodological and other differences between the original trial and reanalysis, to evaluate the independence of authors performing the reanalyses, and to assess whether the reanalysis changed interpretations from the original article about the types or numbers of patients who should be treated. DESIGN: We completed an electronic search of MEDLINE from inception to March 9, 2014, to identify all published studies that completed a reanalysis of individual patient data from previously published RCTs addressing the same hypothesis as the original RCT. Four data extractors independently screened articles and extracted data. MAIN OUTCOMES AND MEASURES: Changes in direction and magnitude of treatment effect, statistical significance, and interpretation about the types or numbers of patients who should be treated. RESULTS: We identified 37 eligible reanalyses in 36 published articles, 5 of which were performed by entirely independent authors (2 based on publicly available data and 2 on data that were provided on request; data availability was unclear for 1). Reanalyses differed most commonly in statistical or analytical approaches (n = 18) and in definitions or measurements of the outcome of interest (n = 12). Four reanalyses changed the direction and 2 changed the magnitude of treatment effect, whereas 4 led to changes in statistical significance of findings. Thirteen reanalyses (35%) led to interpretations different from that of the original article, 3 (8%) showing that different patients should be treated; 1 (3%), that fewer patients should be treated; and 9 (24%), that more patients should be treated. CONCLUSIONS AND RELEVANCE: A small number of reanalyses of RCTs have been published to date. Only a few were conducted by entirely independent authors. Thirty-five percent of published reanalyses led to changes in findings that implied conclusions different from those of the original article about the types and number of patients who should be treated.
IMPORTANCE: Reanalyses of randomized clinical trial (RCT) data may help the scientific community assess the validity of reported trial results. OBJECTIVES: To identify published reanalyses of RCT data, to characterize methodological and other differences between the original trial and reanalysis, to evaluate the independence of authors performing the reanalyses, and to assess whether the reanalysis changed interpretations from the original article about the types or numbers of patients who should be treated. DESIGN: We completed an electronic search of MEDLINE from inception to March 9, 2014, to identify all published studies that completed a reanalysis of individual patient data from previously published RCTs addressing the same hypothesis as the original RCT. Four data extractors independently screened articles and extracted data. MAIN OUTCOMES AND MEASURES: Changes in direction and magnitude of treatment effect, statistical significance, and interpretation about the types or numbers of patients who should be treated. RESULTS: We identified 37 eligible reanalyses in 36 published articles, 5 of which were performed by entirely independent authors (2 based on publicly available data and 2 on data that were provided on request; data availability was unclear for 1). Reanalyses differed most commonly in statistical or analytical approaches (n = 18) and in definitions or measurements of the outcome of interest (n = 12). Four reanalyses changed the direction and 2 changed the magnitude of treatment effect, whereas 4 led to changes in statistical significance of findings. Thirteen reanalyses (35%) led to interpretations different from that of the original article, 3 (8%) showing that different patients should be treated; 1 (3%), that fewer patients should be treated; and 9 (24%), that more patients should be treated. CONCLUSIONS AND RELEVANCE: A small number of reanalyses of RCTs have been published to date. Only a few were conducted by entirely independent authors. Thirty-five percent of published reanalyses led to changes in findings that implied conclusions different from those of the original article about the types and number of patients who should be treated.
Authors: Muthiah Vaduganathan; Amulya Nagarur; Arman Qamar; Ravi B Patel; Ann Marie Navar; Eric D Peterson; Deepak L Bhatt; Gregg C Fonarow; Clyde W Yancy; Javed Butler Journal: Circulation Date: 2017-11-13 Impact factor: 29.690