Use of Patient Health Records to Quantify Drug-Related Pro-arrhythmic Risk.

There is an increasing expectation that computational approaches may supplement existing human decision-making. Frontloading of models for cardiac safety prediction is no exception to this trend, and ongoing regulatory initiatives propose use of high-throughput in vitro data combined with computational models for calculating proarrhythmic risk. Evaluation of these models requires robust assessment of the outcomes. Using FDA Adverse Event Reporting System reports and electronic healthcare claims data from the Truven-MarketScan US claims database, we quantify the incidence rate of arrhythmia in patients and how this changes depending on patient characteristics. First, we propose that such datasets are a complementary resource for determining relative drug risk and assessing the performance of cardiac safety models for regulatory use. Second, the results suggest important determinants for appropriate stratification of patients and evaluation of additional drug risk in prescribing and clinical support algorithms and for precision health.

Introduction

Over the past 10 years there has been an emphasis on use of in silico approaches for cardiac risk assessment. Initially, these computational tools were used to aid pharmaceutical industry decision-making1, 2, 3 and, more recently, by offering an interpretation of in vitro assay data for regulatory purposes. There are good reasons for doing so, most notably an increasing amount (quality and throughput) of in vitro data,, in silico tools,,,,7, 8, 9, 10, 11, 12, 13, 14, 15 supporting research activities,,16, 17, 18 and pressures to adapt an imperfect but apparently successful pair of International Council for Harmonisation of Technical Requirements for Pharmaceuticals for Human Use (ICH) guidance documents, to motivate these efforts.

These guidance documents were introduced in response to a number of drugs being removed from the market in the 1990s and 2000s and were implemented to require testing of compounds for their ability to modulate the human Ether-à-go-go-Related Gene (hERG) potassium channel currents (ICH S7B) and to test compound effects on the QT interval measured from the clinical body surface electrocardiogram (ECG) (ICH E14). Although perceived to be successful in reducing arrhythmia-related (specifically torsades de pointes) drug withdrawal, there was concern that discarding promising therapies on a perceived hERG risk negatively affected novel drug development because these screens result in false positives. To counter this and to incorporate the improved understanding of the mechanisms of proarrhythmia, the Comprehensive In Vitro Proarrhythmia Assay (CiPA) initiative was tasked with defining a new paradigm for cardiac risk assessment using a combination of in vitro screening, stem cell-derived cardiomyocyte tests, and in silico predictions.,

Some of the earlier in silico studies focused on supplementing pre-clinical decisions; for instance, by replacing the need for isolated animal-derived cardiomyocyte experiments., Over time, the output of in silico studies has been challenged to address increasingly more ambitious goals; namely, correlation of simulated cellular action potential biomarkers with the measure between Q wave and T wave (QT interval) in the body-surface ECG from the clinical thorough QT (TQT) study and proarrhythmia., It is important to note that the underlying models have not fundamentally changed in that time, but novel metrics that integrate predictions from single-cell simulations are being considered as surrogate indicators for proarrhythmia., The ambition to extend single-cell simulations to a population-level risk therefore necessitates a thorough evaluation of these in silico tools as a key step toward understanding their utility to predict arrhythmic risk. In a recent study, we showed how a different selection of compounds can have a profound effect on the evaluation score of these models; therefore, a more rigorous effort to establish a fixed and balanced compound set for model evaluation should be considered. Two ongoing initiatives, CiPA and the Japanese induced Pluripotent Stem (iPS) cellsCardiac Safety Assessment (JiCSA) initiative, are attempting to establish a set of in vitro data for model evaluation. Typically, selected evaluation compounds are scored using CredibleMeds evaluation or, in the case of CiPA, interpretation of the CredibleMeds score, including expert assessment that also accounts for clinical experience.

The classification schemes described above and others relevant within the field (such as Redfern category) are designed to simplify risk information, which is a quantitative continuous measure, into a set of qualitative categories. Although this is a valuable (and sometimes necessary) exercise for supporting decision-making, it comes at the cost of losing information and introducing subjectivity, particularly when new information or new compounds are required to be evaluated. This concern is well recognized in medicine, where a desire to dichotomize continuous scales is also prevalent, such as “low” or “high” cholesterol. It has been argued that such dichotomization leads to reduced statistical power in detecting cause and effect. A recent review by Wisniowska and Polak discusses a number of issues that occur when attempting to compare cardiac risk across different classification schemes. One such limitation is how a ranking could be applied, e.g., to previously uncharacterized drugs. The ability to rank compounds in terms of putative risk would be advantageous for ongoing and continual model performance assessment beyond the immediate needs of the CiPA initiative.

To date, consideration of these regulation-led efforts for pro-arrhythmic risk prediction has prioritized focus on reproducibility and variability of the in vitro (i.e., input) data for the models. In this study, we aim to complement those activities by focusing more on the risk classification (i.e., output) scores in the evaluation datasets, and we set out to take advantage of the considerable post-marketing medical use of a broader set of evaluation drugs to establish the frequency of adverse cardiac events. Use of such post-market (i.e., real-world) data sources not only provides an estimate of the rate of adverse events that are observed in a real-life population but, we hypothesize, will also provide a more quantitative and continuous metric for assessing pro-arrhythmic risk.

However, although post-market observational data sources may be a valuable way of gaining insights into routine healthcare practice, they are not without complexity and show variability in patients and in the reporting practices inherent in the real world. One limitation of the data from adverse event databases is that the number of events is not normalized to the number of prescriptions—what we call the denominator problem. In the US Food and Drug Administration (FDA) Adverse Event Reporting System (FAERS), a high incidence of adverse events for a given drug may simply reflect highly prescribed drugs; therefore, statistical methods to identify clinically important adverse events (i.e., when particular adverse events are seen more often than expected) are invaluable for pharmacovigilance. For this study, we used a disproportionality metric of empirical Bayes geometric mean (EBGM) with a threshold of EB05 > 2 as a positive signal commonly used in pharmacovigilance.

To additionally account for the denominator problem, the Truven Health MarketScan® Research Databases were used, which contain individual-level, de-identified healthcare claims information from employers, health plans, hospitals, and Medicare and Medicaid programs. Since their creation in the early 1990s, the MarketScan Databases have grown into one of the largest collections of de-identified patient-level data in the United States. These databases reflect real-world treatment patterns and costs by tracking millions of patients as they travel through the healthcare system, offering detailed information about all aspects of care. Data about individual patients are integrated from all providers of care, maintaining all healthcare utilization and cost record connections at the patient level. Used primarily for research, these databases are fully compliant with United States privacy laws and regulations (e.g., Health Insurance Portability and Accountability Act (HIPAA)).

Until now, many of the existing in silico models were designed and developed to give insights to cardiomyocyte electrophysiology and cellular-level outcomes. Extrapolation to population effects was never the primary design goal, and although approaches have been developed to allow surrogate markers to be evaluated, validation of such markers needs careful consideration. Blinded studies for in silico risk assessment, as performed recently by Zhou et al., are significantly more difficult when the performance or outcomes of the drug effects are defined up front, such as the CiPA classification or CredibleMeds, and a more objective performance metric based on observational data could be used instead.

We set out to test the utility of these so-called real-world datasets to provide insights into the categorization of compounds for proarrhythmic potential to support or refute the clinician-led understanding of risk. Coinciding with the recent General Principles for the Validation of Proarrhythmia Risk Prediction Models, the work was not intended to establish new cardiac safety metrics. Instead, the work was motivated to be complementary and to highlight datasets that should prove to be helpful when appraising the existing metrics, assays, and computational models that have been developed to allow early assessment of cardiac risk potential, particularly in cases of discordance between metrics, and also to stratify individual drug risk in patient subsets.

Results

Cardiac Adverse Events per Year Analysis and Its Regulatory Effect

It is perceived that the regulations in ICH documents S7B and E14 mean that no new drugs have been associated with increased risk of torsades de pointes (TdP) arrhythmias. This study set out to query whether this statement is equivalent to there being no new reports of TdP events. Indeed it would be intuitive to expect that TdP (and other related ventricular conditions) might be observed to have decreased since introduction of these regulations. Therefore, an early aim was to assess TdP incidence and update and extend a previous analysis by Stockbridge et al., who reviewed the annual number of reports received by the FAERS.

The Pharmapendium (Elsevier) tool provides access for querying FAERS reports of TdP events. To recognize that arrhythmia events may be recorded differently clinically than TdP, we selected other heart-related adverse events in addition to TdP, some as sibling Medical Dictionary for Regulatory Activities (MedDRA) terms to TdP (Figure S1) for the years 2000–2015. It is worth noting that FAERS data provide outcomes for each adverse event and that, for cases where “ventricular fibrillation” is reported, approximately 45% result in a fatal outcome, whereas fatality is associated with approximately 12% for reports of TdP. A further significant finding is a reporting delay, observed as a discrepancy between the occurrence date of the adverse event and the reported date to the FDA. Of 56,682 unique cases, 15,931 do not report the event date, and of the remaining cases, only 16,376 (i.e., ~40%) are reported in the same year as the event date, with 2,771 (i.e., ~7%) showing a delay of 5 years or more. Consequently, examining the incidence of cardiac adverse events on a year-by-year basis (Figure 1A) reflects the observation as a drop in the most recent years. For this reason, we also present the data as events per submission year (Figure 1B), where the perceived drop in events is not observed.

Figure 1

Adverse Cardiac Events Being Reported over Time

Number of cardiac-related adverse events per year of event or year of submitted report from FAERS reports extracted from Pharmapendium (Elsevier).

The FAERS data, together with pharmacovigilance analysis, enable the user to spot drug safety signals in a timely manner. However, the database is not without limitations; FAERS does not explicitly account for whether (or how) the drug caused the adverse events or the volume of prescriptions, nor is it exhaustive in covering all possible adverse events. In other words, drugs that are more highly prescribed would be expected to show higher total numbers of events than drugs with the same level of risk that are prescribed less frequently. To partially account for this limitation, a disproportionality metric using the EBGM analysis was used to account for whether a cardiac adverse event rate is disproportionately higher than these background rates. The EB05 is the lower bound of the 95% confidence interval of the EBGM; EB05 values greater than 2 are considered to show a signal and, therefore, a drug-induced risk increase. Table 1 shows the CiPA reference drugs ranked by EB05 value and the corresponding CiPA and CredibleMeds classifications together with the frequently used safety margin built based on the hERG half maximal inhibitory concentration (IC50)/free highest concentration of a drug in the blood (Cmax ) ratio. It is important to recognize that only 6 of 28 CiPA compounds have an EB05 value of less than a positive pharmacovigilance “signal” threshold of 2, which indicates a set of drugs showing a higher propensity for cardiac disorders than other adverse events. Interestingly, ranking of compounds based on their EB05 score (Table 1) shows some discordance between different classification systems. For instance, vandetanib has a low EB05 value but is classified as high risk by CredibleMeds and CiPA. The inverse is also seen with the anti-arrhythmic drug ranolazine, which has a high EB05 value but is ranked “very low” by CiPA. Of the CiPA compounds, it is striking that 8 of the drugs are indicated primarily as anti-anginal or anti-arrhythmic drugs where it might reasonably be expected to see a higher proportion of cardiac adverse events because of patients’ comorbidities. For this reason, we chose to investigate whether an expanded set of drugs (beyond the CiPA list) would provide more drugs with a low EB05 value and cover a more diverse range of drug classes because representing negative drugs is also important for model evaluation.

Table 1

Ranking of CiPA Drugs by Disproportionality (EB05) for Cardiac Adverse Events

Generic Drug Name	EB05 TdP	EB05 VT	EB05 VA	CiPA Classification	CredibleMeds Classification	hERG IC50/Free Cmax Ratio	Drug Class
Ibutilide	218.45	101.022	2.901	high	risk of TdP	3.37	anti-arrhythmic
Azimilide	94.351	1.381	NC	high	NC	11.50	anti-arrhythmic
Bepridil	81.663	38.276	5.155	high	risk of TdP	1.42	anti-anginal
Sotalol	70.355	18.029	14.276	high	risk of TdP	17.2	anti-arrhythmic
Methadone	36.408	3.998	1.87	high	risk of TdP	4.90	opiate
Quinidine	35.667	12.296	2.768	high	risk of TdP	0.92	anti-arrhythmic
Cisapride	30.654	21.801	5.117	intermediate	risk of TdP	8.25	gastro-intestinal stimulant
Terfenadine	24.417	9.397	3.085	intermediate	risk of TdP	0.41	antihistamine
Flecainide	23.364	20.567	4.123	very low	risk of TdP	59.01	anti-arrhythmic
Ranolazine	22.444	4.375	0.205	very low	conditional risk of TdP	2.69	anti-anginal
Dofetilide	20.983	14.397	6.235	high	risk of TdP	4.36	anti-arrhythmic
Droperidol	19.454	4.564	2.899	intermediate	risk of TdP	11.46	anti-psychotic/anti-emetic
Domperidone	18.85	1.468	1.455	intermediate	risk of TdP	810.98	anti-emetic
Astemizole	18.549	15.499	1.965	intermediate	risk of TdP	24.55	antihistamine
Pimozide	17.093	2.332	0.25	intermediate	risk of TdP	16.60	anti-psychotic
Ondansetron	15.333	6.395	1.281	intermediate	risk of TdP	62.62	anti-emetic
Clarithromycin	7.69	3.016	1.898	intermediate	risk of TdP	77.41	antibiotic
Chlorpromazine	5.483	1.78	0.679	intermediate	risk of TdP	64.71	anti-psychotic/anti-emetic
Loratadine	4.873	3.043	0.583	very low	NC	11111.11	antihistamine
Verapamil	3.426	2.381	2.104	very low	NC	7.35	anti-hypertensive
Metoprolol	3.176	3.318	1.955	very low	NC	326.06	adrenoceptor antagonist
Mexiletine	2.649	10.083	3.986	very low	NC	130.11	neuromuscular blocking agent
Diltiazem	2.62	1.443	0.925	very low	NC	210.42	anti-arrhythmic
Risperidone	1.257	0.706	0.543	intermediate	possible risk of TdP	176.99	anti-psychotic, atypical
Nitrendipine	0.618	0.228	NC	very low	NC	50345	anti-hypertensive
Vandetanib	0.546	NC	NC	high	risk of TdP	2.45	anti-cancer
Nifedipine	0.391	0.42	0.76	very low	NC	1754.4	anti-hypertensive
Clozapine	0.191	0.291	0.372	intermediate	possible risk of TdP	7.06	anti-psychotic, atypical
Tamoxifen	0.077	0.172	0.06	very low	possible risk of TdP	284.1	anti-cancer

NC, not classified. The hERG IC50/free Cmax ratio is derived from experimental hERG data and supplemented with prior published values;,36, 37, 38, 39, see Data S3 for full details. Typically, a threshold of 30 is regarded as a cutoff between high- and low-risk drugs. Abbreviations for EB05 values are as follows: TdP, torsades de pointes; VT, ventricular tachycardia; VA, ventricular arrhythmia. CredibleMeds classification and drug classification were correct as of the date of last access (May 22, 2018; http://crediblemeds.org/index.php/login/dlcheck).

Expanded Compound Set for Data Visualizations

To ensure consistency and overlap with previous work, a search was conducted for studies that had already compiled lists of compounds relevant for cardiac risk assessment and model validation.,,, The motivation was to minimize introduction of novel compounds, consolidate prior work, and promote consistency across studies, as discussed recently. Ideally, compounds that have information on ion channel effects, cardiomyocyte action potentials, and ECG effects are most suited for understanding the predictive capacity of pro-arrhythmia models to most reasonably assess their translational capacity.

We composed an initial list of 149 drugs that have a broad range of molecular and in vivo effects. The drugs in our set are comprised of those under study by the JiCSA and CiPA initiatives,, in a recent in vitro assay study and by three other in vitro/in silico combination studies,, and, finally, an unpublished list of 66 reference drugs we judged to give a balance of positive and, critically, negative effects in cardiac ion channel assays. The full list of drugs is given in Data S3, but a number of interesting findings were uncovered in this exercise. Most notably, the overlap between the different studies was low, with no drugs being studied in all of the prior studies; only 4 drugs (quinidine, dofetilide, cisapride, and terfenadine) were studied in 6 of the 7 studies. Furthermore, 89 of the total list of 149 drugs are unique to a single study, meaning that a cross-comparison of different in silico tools is currently difficult to interpret when different sets of compounds are used for evaluation; see, for example, Figure 4 from Davies et al. Therefore, the consensus list of 149 compounds was used as the basis for onward analysis, recognizing that not all of the compounds on this list are approved for clinical use and so would not be identifiable in post-market observational databases.

We now examine how the propensity of cardiac disorders in FAERS reports is distributed in this expanded set of compounds. Figure 2 shows the distribution of EB05 values for TdP and ventricular tachycardia (a sibling MedDRA term for TdP). 28 CiPA compounds are highlighted on the plot according to their risk classification. Again, many of them are presented in the top right quadrant of the EB05 plot, indicating that this set of compounds is unevenly distributed toward more active compounds. We propose that including additional compounds (shown in Figure 2 as non-colored compounds) will facilitate improved evaluation of positive and (equally important) negative signals. In Figure 1, we can see that ventricular tachycardia (VT) is more frequently reported than TdP. Because we see a strong correlation between TdP and VT, VT and similar adverse events (i.e., MedDRA sibling terms to TdP) could potentially be included as part of the overall cardiac risk assessment of a given drug. Broadening the range of terms considered (as done for CredibleMeds) would improve risk sensitivity. This is exemplified by mexiletine, which is classified as low risk by CiPA, and is supported by the marginal EB05 value (EB05 = 2.6) and yet appears to be of higher risk for VT (EB05 = 10.1) or ventricular arrhythmia (EB05 = 4.0). Recognizing that this correlation may simply be representative of co-reporting of the adverse event, we investigated the underlying co-occurrence rate. It was found that the number of VT reports that also co-reported TdP was only approximately 10% (i.e., 1,525 of 15,041). This demonstrates that, typically, cardiac adverse events are reported as one term or another and emphasizes a need to consider a broader scope of adverse outcomes beyond TdP; e.g., VT and ventricular tachyarrhythmia.,

Figure 2

Distribution of the Extended Compound Set for TdP and VT FAERS Reports (Empirica-Derived Data)

(A and B) The axes show EB05 values for the indicated MedDRA code, and each spot represents one of the selected compounds. Horizontal and vertical red lines show EB05 threshold = 2. The same data are shown in both plots with different highlighting that represents (A) CiPA classification and (B) hERG IC50/free Cmax. Arrows indicate drugs showing concordance (Con), discordance (Dis), or unknown (Un) between, e.g., hERG IC50/free Cmax ratio, CiPA, and the EBGM score (also presented in Table 1). A full list of drugs labeled in the order of Data S3 is presented in Figure S2.

See also Data S1.

We analyzed each drug for FAERS reports and also used the MarketScan database (data were collected for the period of January 1, 2009, through December 31, 2014). Because data from healthcare claims are recorded longitudinally along with prescription use, it is possible to normalize events based on drug use (i.e., providing an incidence rate).

Using Electronic Claims Data to Inform Different Outcomes

An optimal strategy for evaluating safety model performance would be to compare against a continuous and objective metric that can be readily calculated for an extended set of compounds. For this purpose, we queried how translation of prior metrics (e.g., hERG IC50/free Cmax ratio and a prior categorization [CiPA risk category]) compares with results from insurance claims records.

The claims data in Figure 3 show a clear trend between total exposure (in patient years) and the incidence of cardiac dysrhythmia, indicating a previously unreported underlying background rate of cardiac dysrhythmia. Color indicates the CiPA score and hERG IC50/Free Cmax ratio. Although some higher-classification drugs (e.g., a CiPA value of high or ratio < 30) appear to stand out above the main cluster, others cannot be readily differentiated from the group.

Figure 3

Translation of Different Ranking Strategies of the Observed Claims Data (from MarketScan)

(A and B) Individual drugs (spots) are overlaid with color for the following reference markers: (A) CiPA ranking and (B) hERG IC50/free Cmax ratio. Drugs with a total patient exposure of less than 100 patient years were excluded from the analysis.

See also Data S2 and S3.

To examine whether measured hERG IC50/free Cmax ratios are concordant with the safety risk, as indicated by the EB05 parameter (from the FAERS database) or the normalized incidence rates (gauged from the MarketScan database), we combined two of these parameters at a time in a conjoint visualization (Figure 4). We use the log-transformed hERG IC50/free Cmax ratio in this case to achieve the effect that higher numerical values represent a higher risk for TdP, which is our targeted endpoint. Based on these graphs, it becomes clear that the hERG measurements coarsely reflect the trend in safety risks signaled by either of the other data sources (FAERS EB05 or MarketScan incidence rate), and although the overall correlation is not very strong (the coefficient of determination R2 = 0.1155 for EB05 and R2 = 0.0573 for the incidence rate), the trends are still significant because of the large number of observations (∗∗p = 0.0016 for EB05 and ∗p = 0.04 for the incidence rate). The prediction interval from a line of best fit shows how hERG measurements actually scatter very widely around this overall trend, which raises concerns regarding use of fixed thresholds on hERG IC50/free Cmax values to stratify compounds with regard to their expected risk of causing TdP events.

Figure 4

Concordance of Safety Signals

(A) logarithmic plots of hERG IC50/free Cmax (triangles) and EB05 for TdP (circles, obtained from FAERS). Compounds were sorted by their EB05 values from large to small.

(B) logarithmic plots of hERG IC50/free Cmax and normalized incidence rate for CD (obtained from MarketScan). Compounds were sorted by normalized incidence rate from large to small.

(A) and (B) Points are color coded by CiPA classification; compounds that were not included in the CiPA list are colored in gray. Note that –log function was applied for the safety margin ratio transformation to account for the compound sorting. A linear regression of hERG IC50/free Cmax values by compound rank (as ordered by the respective other variable) is indicated with a red line and the corresponding 95% prediction interval with a shaded area.

The Importance of Patient Sub-grouping

The striking correlation of exposure to incidence motivated a need to investigate whether drugs with higher incidence are observed in all patient types or whether it is skewed by only a few subtypes. Therefore, a further derivation of the aggregated data and the benefit of working with observational claims data are to explore how patient subtypes affect the rate of cardiac dysrhythmia. For this purpose, we separated each drug into up to 32 individual subtypes based on gender, age (less than 18, between 18 and 44, 45–64, and older 65 years), and degree of comorbidities. Comorbidities were evaluated using the Charlson index, which accounts for a patient’s pre-existing conditions and, accordingly, provides a weighted analysis, and binned into 4 groups (score = 0, 1, 2, or ≥3). It is worthwhile to note that not all drugs showed the full range of these combinations, reflecting that not all drugs are prescribed for all subtypes; e.g., vandetanib, an anti-cancer agent, is unlikely to have been prescribed for lower Charlson index patient subgroups. This rich dataset provides the previously unexplored ability to query our pre-existing assumptions about the correlation with drug risk classification and observed levels of pro-arrhythmia. This is critical to ensure that we allow unknown influences in addition to ion channel inhibition as factors predicting pro-arrhythmic potential. Identified factors such as age and comorbidities could be subsequently incorporated more explicitly into mathematical models or implicitly via a population-type approach, as suggested previously.,,

Exploring the different subsets also enabled us to make an estimate of the background rate of cardiac dysrhythmia within each of the different subgroups. This is important for understanding the patient context of intended drug risk because not all drugs elicit an adverse response in all patient subtypes. Therefore, an understanding of the expected rate of cardiac dysrhythmia (CD) in each different patient subtype should offer an alternative mechanism for categorizing drug risk, given the variable baseline of incidence, and, hence, allow more stratified treatment options. To carry out this analysis, we excluded drugs where total use was less than 100 patient years (as this tends to skew the incidence rate and is not sufficiently representative). From this, the average incidence rate across drugs for each age group and comorbidity group was calculated (Table S2). In general, we observe that older patient subgroups and those in which the Charlson comorbidity score was greater than 3 tend to show the highest incidence rates compared with subgroups where no comorbidities were identified.

Drugs could be broadly be categorized into 3 distinct types of profiles: those that showed an elevated incidence of proarrhythmia regardless of patient subgroup, those showing a normal (or lower) incidence of CD regardless of subgroup, and those that show a differential response between patient subgroups. Three exemplar drugs—the antiarrhythmic flecainide, the antibiotic moxifloxacin, and the antidepressant desvenlafaxine—are shown in Figure 5. In the case of flecainide, for each patient subgroup, a higher rate of CD incidence was observed than the aggregated value of 23.0. For moxifloxacin, the subgroups are highly variable for incidence rate, whereas for desvenlafaxine, the majority of subgroups are near or below this background rate. It is interesting to note that the EB05 values for these drugs (flecainide, 23.36; moxifloxacin, 6.6; desvenlafaxine, 0.13) correlate well with the observed claims data and indicate that EB05 may have merit as a useful metric for quantifying proarrhythmia, particularly when other classifications schemes are missing, as in the case of desvenlafaxine.

Figure 5

Stratification of Patient Subtypes Shows Differences in the Incidence of CD

A scatterplot of CD rates shows incidence rate differences between patient subgroups (split into up to 32 individual subtypes based on age, gender, and comorbidity index) for 3 exemplar drugs of high incidence, mixed incidence, and low incidence, derived from the MarketScan claims database. Colors represent different age groups for patients: green, patients younger than 18 years; yellow, between 18 and 45 years; orange, between 45 and 65 years; red, over 65 years. Dashed lines represent the mean incidence rates across all drugs for these age groups together with the overall mean incidence rate (black dashed line), as seen in Table S2.

A further observation with moxifloxacin and flecainide was how the subgroup incidence rate was highly correlated with the age of the patient (inversely for flecainide), and we chose to investigate whether this was related to isolated drugs or a more general finding. Interestingly, for other antiarrhythmics (amiodarone, disopyramide, dofetilide, dronedarone, quinidine, and sotalol) and antibiotics (azithromycin, ciprofloxacin, clarithromycin, erythromycin, metronidazole, and pentamidine) in the evaluation set, a very similar pattern of age dependency was observed. This observation indicates that it could be related to the class of drugs or even the underlying medical condition for which the drugs are being rather than a specific action of the drug. This could have implications for how drugs are classified for cardiac risk; patient age could be a strong predictor for risk classification. This also suggests how appropriate stratification of patient subsets could be useful in prescribing and clinical support algorithms (i.e., to avoid prescribing to subtypes most at risk).

Future Metrics for Classifying Drug Risk

In this study, we considered how post-market datasets may complement and augment our current assumptions regarding drug-induced cardiac risk. When considering a far wider selection of drugs than previous studies, together with a wider portfolio of complementary data sources, we can challenge or confirm our empirical assessment of cardiac risk, which can potentially lead to an improvement in our evaluation of in silico and/or in vitro models. However, it is apparent that no single marker (i.e., the hERG IC50/free Cmax ratio), will successfully categorize each drug. Table 2 shows a selection of drugs for which different classifications overlaid with claims data from MarketScan demonstrate concordance or discordance between classification systems and also where opportunities for classifying unknown drugs can be used. This is well recognized by the Arizona Center for Education and Research on Therapeutics (AZCERT) group, which has developed a method (adverse drug event causality analysis [ADECA]) for stratifying risk based on multiple inputs, including FAERS, clinical evidence of TdP and hERG inhibition, and the QTDrugs list. The ADECA process performs this well by considering multiple data points from 4 different sources, including biomedical literature, drug labels, and adverse event reports, when classifying a risk score. However, the list is limited in its utility for validation and benchmarking because lack of categorization of a drug cannot be used as an equivalent to “no risk,” and many drugs remain uncategorized, partially because of incomplete data or a lag in the report times of FAERS reports or literature evidence. There remains a need for a systematic, transparent, and (preferably) automated approach to quantify cardiac risk for a chemical. This would ideally build on and develop work already done to provide transparent and available models for cardiac risk assessment; e.g., by the FDA (https://github.com/FDA/CiPA) and also open-source platform AP-Portal, a cardiac electrophysiology simulator based on the published interface developed by Williams and Mirams. We propose that electronic health care records should be considered together with other risk factors, such as patient comorbidities, co-medications, and lifestyle factors (among others), in line with the current healthcare digitalization trend within the next decade.

Table 2

Selected Drugs Exhibiting Concordance or Discordance across Different Risk Classification Schemes or Drugs that Are Currently Uncategorized and where Novel Quantitative Metrics Could Be Supportive

Drug	CD incidence	Incidence Rate per 1,000 Years	Delta from Background Rate	CiPA Classification	CredibleMeds	hERG IC50/Free Cmax Ratio	EB05	Drug Class
Concordant Drugs
Dofetilide	2,749	157.58	134.58	high	risk of TdP	4.36	20.98	anti-arrhythmic
Loratadine	2,016	13.07	−9.93	very low	N/A	11,111.1	4.87	antihistamine
Nifedipine	8,563	20.27	−2.73	very low	N/A	1,754.4	0.39	calcium channel blocker
Quinidine	421	87.75	64.75	high	risk of TdP	0.92	35.67	anti-arrhythmic
Sotalol	13,186	124.65	101.65	high	risk of TdP	17.2	70.36	anti-arrhythmic
Discordant Drugs
Amiodarone	21,788	162.2	139.2	N/A	risk of TdP	737.1	21.35	anti-arrhythmic
Methadone	1,662	19.73	−3.27	high	risk of TdP	4.9	36.41	opiate
Mexiletine	1,250	280.45	257.45	very low	N/A	130.11	2.65	anti-arrhythmic
Paliperidone	119	10.13	−12.87	N/A	possible risk of TdP	87.0	0.57	anti-psychotic
Risperidone	4,066	18.47	−4.53	intermediate	possible risk of TdP	176.99	1.26	anti-psychotic
Unclassified Compounds
Desvenlafaxine	2,222	8.26	−14.74	N/A	N/A	N/A	0.13	antidepressants
Propafenonea	6,643	135.17	112.17	N/A	N/A	N/A	3.38	anti-arrhythmic

Propafenone was added (March 1, 2018) to the CredibleMeds listing as having a conditional risk for TdP.

Discussion

The purpose of our study was to highlight that cardiac risk decision-making requires us to not only use empirical knowledge of drug use but also to augment it with larger observational post-market data (e.g., FAERS, health insurance claims, and electronic patient healthcare records) that are able to support or refute the clinician-led understanding of risk. To the same extent that high quality input data are a necessity for meaningful training of in silico models (e.g., the model parameters), so too must high-quality outcome data be considered for the models’ credibility or for model validation exercises. Consideration of the outcome data is critical for the model validation exercise to ensure a model that is best in class for arrhythmia prediction and compound stratification. Similar challenges have been reported before; for instance, for classification of hepatotoxicity or prediction of cancer driver genes, where the gold standard or truth is unknown. A potential consequence of failing to consider outcomes is that false confidence can be attributed to the selected model and, therefore, subsequent predictions of novel compounds.

In this study, we chose to supplement and review the existing standard approaches (e.g., hERG IC50/free Cmax safety margin ratio and CiPA classification ranking) by considering how datasets that account for the incidence of proarrhythmia derived from the real-world setting can be used to support ongoing evaluation of proarrhythmic risk and offer an opportunity to test our prior assumptions regarding cardiac safety outcomes in patients.

An important motivation for this study was to better understand the possible limitations of the current models to help shape the direction of future development. Whether this means including additional biological details to better represent patient variability or using more empirical models should be an ongoing challenge for the computational biology community, who are likely to be beneficiaries from the extensive datasets being generated within the CiPA and JiCSA initiatives to support these efforts. An important aspect of CiPA and similar initiatives is to consider how to perform an ongoing evaluation of models as new data emerges; e.g., post-market safety signals. In this study, we suggest the types of datasets and possible metrics that would support this effort. Therefore, it was important to carefully consider the data source for its appropriateness for validation of in silico predictions. I It is equally essential that we recognize that lack of a strong signal in the post-market and insurance claims data for drugs with a previously identified risk of pro-arrhythmic potential should challenge us to re-evaluate our risk categorizations.

Observational claims data sources offer great potential for being able to supplement our existing data resources, such as biomedical literature or clinical trial data repositories (e.g., https://clinicalstudydatarequest.com/). However, there are still a number of limitations of these data sources that should be overcome to improve the relevance; these are discussed briefly here. For instance, for this study, we include an incidence rate for “drug-burdened” patients; i.e., we can only include patients who have visited their medical professional, and the calculation of a background rate in healthy patients is typically not collected. However, the opportunity of mobile health (e.g., the AliveCor device) may allow improved understanding of the true background in an otherwise healthy population. In a recent study, Hingorani et al. estimate that 13 healthy volunteers in 1,000 (1.3%) would be expected to show non-sustained VT (NSVT) over a 24-h ECG recording period. Solomon et al. look at arrhythmia detection beyond 24 h and conclude that the incidence of background arrhythmia could be higher still, with 18.3% incidence of NSVT in 128,401 continuously monitored patients over 14 days.

Interoperability across the different post-market datasets (e.g., between FAERS and claims data) is hampered by the different clinical coding dictionaries that are used to identify a medical event. For FAERS, events are represented by the MedDRA dictionary, whereas claims data use the International Classification of Diseases, Ninth Revision, Clinical Modification (ICD-9-CM) vocabulary. This means that, for a given event of TdP, although this can be represented as such in FAERS, an equivalent term is not available in the ICD-9-CM vocabulary and, hence, would be recorded elsewhere. For the purpose of this study, we made the assumption that CD in ICD-9-CM would be an approximation of cases of TdP (and related arrhythmias) but would also encompass other events. However, because focusing solely on whether a drug has caused a TdP event might limit our understanding of a more complete cardiac safety concern, the broader term CD may be a more suitable outcome measure.

Comedications (sometimes referred to as polypharmacy) are a frequent issue and another confounding factor with adverse event reports. Recently, a study investigated the role of multiple-ion-channel testing in determining the mechanistic reasons of loperamide’s proarrhythmic potential in overdose situations. Many of the reported overdoses cases, however, also exhibited polypharmacy, including drugs classified by CredibleMeds in many of the subjects in which loperamide was a cause. This polypharmacy observation is further supported by our analysis of FAERS reports in which loperamide is rarely a primary causative drug taken in isolation. This can make it particularly challenging with regard to identifying the primary drug and/or underlying genetic mutations responsible for the adverse event and identifying the contributing effects of these comedications (and comorbidities). Because this is the case, improvements in how we assess drug risk in the context of typical comedications (particularly, e.g., Cytochrome P450 (CYP) inhibitors and other ion channel inhibitors) would be worthwhile and further support a need for in silico or clinical decision support systems such as CredibleMeds.

Ideally, an understanding of drug-induced proarrhythmia cases rather than drug-associated cases would provide the ideal calibration for computational modeling based on ion channel screening data and in silico predictions. This has been advocated previously by other reporters; for example, Mason recently proposed a need for formal validation with patient outcomes to move away from the current “surrogate” (e.g., hERG inhibition or QTc prolongation) model of cardiac risk. However, studies tackling the epidemiology of drug-induced arrhythmia are limited in the number of patients and cases studied; the Berlin Pharmacovigilance Center (PVZ-FAKOS) and the Drug-induced Arrhythmia Risk Evaluation (DARE) studies are recent examples. Despite their small size (130 cases in DARE and 58 in the PVZ-FAKOS study), there is useful understanding resulting from these studies, notably identification of drugs with no previous classification risk of QTc prolongation or TdP, such as propafenone. This observation clearly shows how existing classifications (CredibleMeds in this case) can be misleading for our assumptions regarding proarrhythmic potential; a case of “the unknown unknowns” (i.e., a negative CredibleMeds classification) is not equivalent to no-risk. These studies point to further improving our view of drug-induced arrhythmias. However, these studies are difficult and costly to conduct; therefore, the observational datasets (e.g., based on claims data) offer an excellent bridging study.

Reporting dynamics and quality should be considered. A pharmacovigilance signal that partly informs the CredibleMeds classification can and does change over time, particularly for newer-to-market drugs, as novel observations are made with increasing clinical use. Hence, the stability and appropriateness of these rankings will affect in silico model selection and validation exercises; i.e., the optimum model may succeed at a later time for no reason other than a change in risk evaluation of one or more of the validation study drugs. The FAERS datasets, for instance, are predominantly based on United States reports (approximately 70% in 2014) and underreporting of adverse events (e.g., 80% underreporting of serious adverse drug reactions) has been reported previously. The reporters to FAERS are also highly mixed. When we considered 6,470 individual TdP events, 34% did not give a primary reporter occupation, and only 28% were from a physician. This implies that more than two-thirds of TdP reports are reported by individuals other than a physician; this motivated us to consider insurance claims data to reduce bias as a result of the reporter. In addition, FAERS reports, perhaps linked to the reporter, can be influenced significantly by external events, such as safety alerts and labeling of the product with indications of cardiac events. In our sample set, we identified 55 drugs with a product label containing a cardiac warning (data obtained from CredibleMeds). The median EB05 value (for TdP) for drugs with a label warning for TdP was 7.84, whereas drugs that did not specifically mention TdP was 1.48. Although a product label can result in overreporting and underreporting of events, it is nevertheless consistent with the hypothesis that drug warning labels for TdP can cause tendency in the healthcare community for overreporting events. A number of drugs are highly reported for cardiac adverse events within a short period of time and can potentially skew the data. We therefore recognized a need for augmenting any reporter-led datasets because of these biases, which would equally apply to FAERS, World Health Organization, and European Medicines Agency adverse events with insurance claims datasets.

Full coverage across datasets (e.g., data missing for hERG IC50, drug Cmax, CredibleMeds analysis) or prior classifications makes comprehensive cross-comparison more difficult and limits the number of drugs for which comparisons can be made. However, even with these limitations, this study captures a number of drugs for which data across the different categories are present; 57 drugs, for example, have information from claims (MarketScan) data, hERG IC50 data, or EB05 (FAERS pharmacovigilance) data, of which only 36 have a corresponding CredibleMeds classification. We advocate for continual assessment and experiments that help improve this set of 57 drugs, and this should be a priority for further studies and developments in this area. One outcome of the ongoing regulatory initiatives is that multiple experimental values, rather than single IC50 records, will be generated and, therefore, will provide an understanding of experimental variation that can be subsequently modeled to better represent experimental uncertainty.

It has to be noted that it is not possible at this stage to gauge the biases that are present in either data source, so a weak correlation between different measures just reiterates a general concern regarding blindly trusting the available data. The finding does not challenge any specific parameter, so in practice, it would be up to the prior assumptions of the researcher to properly weight the sources of evidence. One could, for example, assume that a set of hERG channel binding values obtained under constant conditions in one lab is much harder to question than any observational dataset that comes with plenty of potential biases. From Figure 4, it can also be inferred that the CiPA classification of compounds is backed by other measures, mostly for the high-risk category, whereas separation between a medium- and low-risk class is much harder to justify, especially when looking at the reported incidence rates. If real, then this finding would have notable implications for construction of mathematical prediction models hinging on those labels.

An emergent outcome of this study is to demonstrate the potential for a more general utility of post-market datasets for modeling and simulation as a result of improving data access and availability to more generally support systems pharmacology/biology model calibration and evaluation. Finally, the data from post-market sources offer an opportunity to attribute drug risk to many of the drugs uncategorized by CiPA, CredibleMeds, or Redfern. As an example, propafenone (indicated in Figure 4) has recently been described as causing 3 proarrhythmia cases; this was subsequently added (March 1, 2018) to the CredibleMeds listing as having a conditional risk for TdP. The disproportionality index calculated on FAERS data shows a value of more than 2.0, and using the incidence data from MarketScan data in Figure 4 also indicates that the drug resides on the upper portion of the scatterplot, consistent with the signal from FAERS. We anticipate that this work can also be valuable for drug repurposing and repositioning, particularly when the benefit/risk is changed significantly for the new proposed indication. As a method for providing quantitative, transparent proarrhythmic risk, these datasets are additional tools to support clinical decision-making and risk/benefit analysis.

These datasets are still somewhat nascent in their utility to support the field of quantitative systems pharmacology, but by developing methods to show how they can be used, we also show how future collection of real-world health datasets can be aligned with supporting risk management. We hope this will encourage experimentalists, data scientists, and clinicians to work together to develop a transparent model-driven approach based on FAIR (findable, accessible, interoperable, and reusable) data standards. The framework should enable scientists, sponsors, and decision-makers to quantitatively evaluate the probability of success of new medicines in a better computer-augmented and human-rendered way that can support more nuanced and patient-specific prescribing.

Limitations of Study

As discussed above, this work is not without limitations, the most significant being the difference of correlation versus causation of drug-induced pro-arrhythmia. Being able to definitively state that an arrhythmic event is the sole result of a prescribed drug is hard, and we typically use surrogates such as prolonged QTc. 2 recent studies, PVZ-FAKOS and DARE, have successfully addressed this issue but are limited in size of patient population. In our study, we looked at a fixed time period with patient health records following commencement of a new drug prescription to minimize the risk of confounders. Additionally, there was a lack of consistency across the different post-market datasets; i.e., between the FAERS and MarketScan data for arrhythmia events because of differences in coding dictionaries (Table S1). We therefore used CD from ICD-9 as a surrogate for the MedDRA-coded events in FAERS; e.g., VT or TdP. The intent of this study is to demonstrate what can be achieved with current datasets.

STAR★Methods

Key Resources Table

Resource Availability

Lead Contact

Further information and requests for resources should be directed to and will be fulfilled by the Lead Contact, Liudmila Polonchuk (liudmila.polonchuk@roche.com)

Materials Availability

This study did not generate any unique reagents/materials

Data and Code Availability

The published article includes all datasets generated during this study

Experimental Model and Subject Details

hERG testing

To improve consistency and minimize lab-to-lab variance we chose to profile the electrophysiological effects of compounds against hERG ourselves and collect free Cmax concentrations of drugs using, where possible a primarily single source. Assessment of pro-arrhythmia algorithms will be most efficient if the compound set includes both positive and negative response compounds in order to ensure an adequate assessment of a model’s positive and negative predictive values.

Compounds

Reference drugs were purchased from commercial vendors. Selection of test concentrations for each compound was done based on the hERG potency data and the solubility in the extracellular solution. Stock solutions of compounds were freshly prepared in DMSO. Test solutions were made such that solvent concentrations were kept constant throughout the experiment (0.1%).

Cell culture

The CHO crelox hERG cell line (ATCC reference Nr. PTA-6812, female Chinese hamster cells) was generated and validated at Roche. Ready-to-use frozen instant CHO-hERG cells are cryopreserved at Evotec (Germany). For the experimental use, the vials with cryopreserved cells are thawed at 37°C, washed with the pre-warmed IMDM cell culture medium (GIBCO Life Technologies, USA) and re-suspended in the extracellular solution.

Solutions

The extracellular solution contains (in mM): NaCl 150; KCl 4; CaCl2 1; MgCl2 1; HEPES 10; pH 7.2-7.4 with NaOH, osmolarity 290-330 mOsm. The internal solution contains (in mM): KCl, 10; KF, 100; NaCl, 10; HEPES, 10; EGTA, 20; pH = 7.0-7.4 with KOH, osmolarity 260-300 mOsm.

Electrophysiology

The hERG test is performed using automated patch clamp system SynchroPatch® 384 (Nanion Technologies GmbH, Germany) at 35-37°C following the experimental procedure described previously.

Subjects

Patients were selected by exposure to either of a list of drug compounds (from NDC codes) used for this study from 2009 −2014. In total, the cohort included 49,421,340 patients, of which 43.6% were male (mean age 36.74 years) and 56.4% female (mean age 38.05 years). All enrolment records and inpatient, outpatient, ancillary, and drug claims were collected.

Quantification and Statistical Analysis

Datasets used in this study

The two post market datasets used in this study show different strengths and limitations and hence were both necessary for the purpose of the included work, a summary of the major differences is provided in Table S1.

FAERS

The FDA Adverse Event Reporting System database (FAERS) is based upon voluntary reports of post marketed drug safety. It is a useful resource for pharmacovigilance and monitoring of potential signals that can be apparent only when larger numbers of patients are exposed to a drug, particularly for rare events such as ventricular arrhythmias. Data for this study was from FAERS (since Nov 1997) up to March 31, 2015. EB05 values were calculated from the FAERS data using the Empirica Signal version 8.1 from Oracle. The cumulative gamma distribution function can be used to obtain percentiles of the posterior distribution of λ. The equation was as follows: EB05ij = Solution to: Prob(λ < EΒ05 | Nij, θ) = 0.05; where I and j represent the drug and event under study. Duplicate reports as identified by Oracle were excluded from the analysis. MedDRA version 18.0 was used for the purpose of this study.

Truven Health MarketScan® Commercial and Medicare Supplemental Database

Data used for the analysis were derived from the Truven Health MarketScan® Commercial Claims and Encounters and Medicare Supplemental and Coordination of Benefits research data bases (Truven Health Analytics, Ann Arbor, Mich.) for the period January 1, 2009, through to December 31, 2014. These databases represent the health services of approximately 170million employees, dependents, and retirees in the United States with primary or Medicare supplemental coverage through privately insured fee-for-service, point-of-service, or capitated health plans.

Index Date

The index date for patients was the date they met the criteria of exposure to selected treatments according to the inclusion criteria.

Exposure period (time at risk)

Claims supply days were used to determine exposure; if a claim had a missing or zero day supply the median day supply was assigned corresponding to the drug name and route of administration. Exposure was defined as the time from the first treatment claim until the last treatment claim + median supply in the enrolment period. If two consecutive treatment claims in the exposure period were more than two times the median supply days apart, this was considered a gap and treatment exposure was stopped at the last treatment claim prior the gap + median supply days.

Outcomes

The present study assessed the incidence of Cardiac dysrhythmia from inpatient and outpatient claims using ICD-9 diagnosis codes.

Statistical Analysis

The incidence rates (per 1000 person-years, with 95% confidence intervals [CIs] calculated using the Poisson regression) of any event were computed as the number of patients with ≥ 1 event of interest divided by the sum of the person-time at risk until the first event, or total exposure if no event occurred. The follow-up data were censored at either the date of the first occurrence of the cardiac event for patients with the event of interest or the date corresponding to the end of their follow-up period (disenrollment or end of exposure period).

REAGENT or RESOURCE	SOURCE	IDENTIFIER
Chemicals, Peptides, and Recombinant Proteins
CaCl2	Acros	AC349615000; CAS Number 10043-52-4
EGTA	Sigma-Aldrich	0396; CAS Number 67-42-5
HEPES	Applichem	A1069,0500; CAS Number 7365-45-9
KCl	Acros	193780010; CAS Number 7447-40-7
KF	Acros	449148; CAS Number 7789-23-3
KOH	Sigma-Aldrich	417661; CAS Number 1310-58-3
NaCl	Merck	106404; CAS Number 7647-14-5
NaOH	Sigma-Aldrich	72068; CAS Number: 1310-73-2
MgCl2	Sigma-Aldrich	442611; CAS Number 7791-18-6
ALPRAZOLAM	TRC	A575650; CAS Number 125316-83-8
AMIODARONE	Sigma-Aldrich	A-8423; CAS Number 19774-82-4
ASTEMIZOLE	Sigma-Aldrich	A-6424; CAS Number 68844-77-9
AZIMILIDE	TRC Canada	A926950; CAS Number149888-94-8
BEPRIDIL	Sigma-Aldrich	B-5016; CAS Number 74764-40-2
CAPTOPRIL	Sigma-Aldrich	21751; CAS Number 62571-86-2
CHLORPROMAZINE	AKSCi	J11680; CAS Number 69-09-0
CIPROFLOXACIN	LKT Labs	C3262; CAS Number 85721-33-1
CISAPRIDE	Tocris	1695; CAS Number 81098-60-4
CITALOPRAM	USP	1134233; CAS Number 59729-32-7
CLARITHROMYCIN	LKT Labs	C4502; CAS Number 81103-11-9
CLOZAPINE	Sigma-Aldrich	C6305; CAS Number 5786-21-0
DASATINIB	Cayman Chemical	11498; CAS Number 302962-49-8
DILTIAZEM	Sigma-Aldrich	D2521; CAS Number 33286-22-5
DOFETILIDE	Cayman Chemical	Cayman/15045; CAS Number 115256-11-6
DOXORUBICIN	Cayman Chemical	15007; CAS Number 25316-40-9
DULOXETINE	Roche	RO4500720-000-001
ERLOTINIB	Cayman Chemical	10483; CAS Number 183321-74-6
ERYTHROMYCIN	ICN Biomedicals	1890197; CAS Number 114-07-8
FLECAINIDE	Sigma-Aldrich	F-6777; CAS Number 54143-56-5
FLUOXETINE	USP	1279804; CAS Number 56296-78-7
GREPAFLOXACIN	Roche	RO0661290-000-001
HALOPERIDOL	Sigma-Aldrich	H1512; CAS Number 52-86-8
IBUPROFEN	Euro Pharma	I0020000; CAS Number 15687-27-1
IBUTILIDE	TargetMol	T6541; CAS 122647-32-9
IMATINIB	Sigma-Aldrich	SML1027; CAS Number 220127-57-1
LORATADINE	Fluka	PHR1376; CAS Number 79794-75-5
METHADONE	Roche	RO0021631-000-001
METOPROLOL	Fluka	80337; CAS Number 56392-17-7
MEXILETINE	Sigma-Aldrich	M2727; CAS Number 5370-01-4
MOXIFLOXACIN	Oakwood Products	079434; CAS Number 186826-86-8
NIFEDIPINE	Calbiochem	481981; CAS Number 21829-25-4
NITRENDIPINE	Sigma-Aldrich	N144; CAS Number 39562-70-4
OLANZAPINE	TRC Canada	O253750; CAS Number 132539-06-1
ONDANSETRON	Roche	RO0418459-000-001
PENTAMIDINE	Sigma-Aldrich	P0547; CAS Number 140-64-7
PHENYLEPHRINE	Sigma-Aldrich	P6126; CAS Number 61-76-7
PIMOZIDE	Sigma-Aldrich	P1793; CAS Number 2062-78-4
QUINIDINE	Sigma-Aldrich	Q-0750; CAS Number 6151-40-2
RANOLAZINE	Kemprotec Limited	CAS Number 95635-55-5
RISPERIDONE	USP	1604654; CAS Number 106266-06-2
SPARFLOXACIN	Fluka	56968; CAS Number 110871-86-8
TAMOXIFEN	Sigma-Aldrich	T5648; CAS Number 10540-29-1
TERFENADINE	Sigma-Aldrich	T9652; CAS Number 50679-08-8
THIORIDAZINE	Sigma-Aldrich	T9025; CAS Number 130-61-0
VANDETANIB	LC Laboratories	V-9402; CAS Number 443913-73-3
VERAPAMIL	Sigma-Aldrich	V4629; CAS Number 152-11-4
ZIPRASIDONE	Roche	RO0724012-000-001
Experimental Models: Cell Lines
Hamster: CHO cells	ATCC	PTA-6812
Software and Algorithms
Empirica Signal version 8.1	Oracle ® Health Sciences	https://docs.oracle.com/cd/E60407_01/index.htm
MedDRA version 18.0	ICH	https://www.meddra.org/how-to-use/support-documentation/english/welcome
MarketScan® Commercial Claims and Encounters and Medicare Supplemental and Coordination of Benefits	Truven Health Analytics	https://www.ibm.com/watson-health/about/truven-health-analytics
R statistical software	R Development Core Team	https://www.r-project.org/
SAS®	SAS	https://www.sas.com/en_us/company-information/profile.html#