OBJECTIVE: To evaluate the performance of a disproportionality design, commonly used for analysis of spontaneous reports data such as the FDA Adverse Event Reporting System database, as a potential analytical method for an adverse drug reaction risk identification system using healthcare data. RESEARCH DESIGN: We tested the disproportionality design in 5 real observational healthcare databases and 6 simulated datasets, retrospectively studying the predictive accuracy of the method when applied to a collection of 165 positive controls and 234 negative controls across 4 outcomes: acute liver injury, acute myocardial infarction, acute kidney injury, and upper gastrointestinal bleeding. MEASURES: We estimate how well the method can be expected to identify true effects and discriminate from false findings and explore the statistical properties of the estimates the design generates. The primary measure was the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. RESULTS: For each combination of 4 outcomes and 5 databases, 48 versions of disproportionality analysis (DPA) were carried out and the AUC computed. The majority of the AUC values were in the range of 0.35 < AUC < 0.6, which is considered to be poor predictive accuracy, since the value AUC = 0.5 would be expected from mere random assignment. Several DPA versions achieved AUC of about 0.7 for the outcome Acute Renal Failure within the GE database. The overall highest DPA version across all 20 outcome-database combinations was the Bayesian Information Component method with no stratification by age and gender, using first occurrence of outcome and with assumed time-at-risk equal to duration of exposure + 30 d, but none were uniformly optimal. The relative risk estimates for the negative control drug-event combinations were very often biased either upward or downward by a factor of 2 or more. Coverage probabilities of confidence intervals from all methods were far below nominal. CONCLUSIONS: The disproportionality methods that we evaluated did not discriminate true positives from true negatives using healthcare data as they seem to do using spontaneous report data.
OBJECTIVE: To evaluate the performance of a disproportionality design, commonly used for analysis of spontaneous reports data such as the FDA Adverse Event Reporting System database, as a potential analytical method for an adverse drug reaction risk identification system using healthcare data. RESEARCH DESIGN: We tested the disproportionality design in 5 real observational healthcare databases and 6 simulated datasets, retrospectively studying the predictive accuracy of the method when applied to a collection of 165 positive controls and 234 negative controls across 4 outcomes: acute liver injury, acute myocardial infarction, acute kidney injury, and upper gastrointestinal bleeding. MEASURES: We estimate how well the method can be expected to identify true effects and discriminate from false findings and explore the statistical properties of the estimates the design generates. The primary measure was the area under the curve (AUC) of the receiver operating characteristic (ROC) curve. RESULTS: For each combination of 4 outcomes and 5 databases, 48 versions of disproportionality analysis (DPA) were carried out and the AUC computed. The majority of the AUC values were in the range of 0.35 < AUC < 0.6, which is considered to be poor predictive accuracy, since the value AUC = 0.5 would be expected from mere random assignment. Several DPA versions achieved AUC of about 0.7 for the outcome Acute Renal Failure within the GE database. The overall highest DPA version across all 20 outcome-database combinations was the Bayesian Information Component method with no stratification by age and gender, using first occurrence of outcome and with assumed time-at-risk equal to duration of exposure + 30 d, but none were uniformly optimal. The relative risk estimates for the negative control drug-event combinations were very often biased either upward or downward by a factor of 2 or more. Coverage probabilities of confidence intervals from all methods were far below nominal. CONCLUSIONS: The disproportionality methods that we evaluated did not discriminate true positives from true negatives using healthcare data as they seem to do using spontaneous report data.
Authors: Abraham G Hartzema; Christian G Reich; Patrick B Ryan; Paul E Stang; David Madigan; Emily Welebob; J Marc Overhage Journal: Drug Saf Date: 2013-10 Impact factor: 5.606
Authors: A Bate; M Lindquist; I R Edwards; S Olsson; R Orre; A Lansner; R M De Freitas Journal: Eur J Clin Pharmacol Date: 1998-06 Impact factor: 2.953
Authors: Martijn J Schuemie; M Soledad Cepeda; Marc A Suchard; Jianxiao Yang; Yuxi Tian; Alejandro Schuler; Patrick B Ryan; David Madigan; George Hripcsak Journal: Harv Data Sci Rev Date: 2020-01-31
Authors: Martijn J Schuemie; Rosa Gini; Preciosa M Coloma; Huub Straatman; Ron M C Herings; Lars Pedersen; Francesco Innocenti; Giampiero Mazzaglia; Gino Picelli; Johan van der Lei; Miriam C J M Sturkenboom Journal: Drug Saf Date: 2013-10 Impact factor: 5.606
Authors: Paul E Stang; Patrick B Ryan; J Marc Overhage; Martijn J Schuemie; Abraham G Hartzema; Emily Welebob Journal: Drug Saf Date: 2013-10 Impact factor: 5.606
Authors: Patrick B Ryan; Paul E Stang; J Marc Overhage; Marc A Suchard; Abraham G Hartzema; William DuMouchel; Christian G Reich; Martijn J Schuemie; David Madigan Journal: Drug Saf Date: 2013-10 Impact factor: 5.606
Authors: Susan Colilla; Elad Yom Tov; Ling Zhang; Marie-Laure Kurzinger; Stephanie Tcherny-Lessenot; Catherine Penfornis; Shang Jen; Danny S Gonzalez; Patrick Caubel; Susan Welsh; Juhaeri Juhaeri Journal: Drug Saf Date: 2017-05 Impact factor: 5.606
Authors: Patrick B Ryan; Martijn J Schuemie; Emily Welebob; Jon Duke; Sarah Valentine; Abraham G Hartzema Journal: Drug Saf Date: 2013-10 Impact factor: 5.606