Comparison of Two Highly Automated ECG Algorithms for Detection of Drug-Induced Cardiac Ion Channel Block.

US Food and Drug Administration (FDA) investigators recently demonstrated in a crossover study that early (J-Tpeak c) and late (Tpeak -Tend ) repolarization duration can differentiate selective potassium block with a high arrhythmia risk from multichannel block with lower risk in subjects receiving dofetilide, verapamil, quinidine, or ranolazine. The purpose of this study was to determine if the findings by the FDA using their published software algorithm could be corroborated using an alternative software algorithm for the same metrics and to determine if methodological differences resulted in clinically meaningful differences in interpretation. Exposure-response relationships computed with linear mixed effects models and mean maximal effects on ECG intervals measured by the two algorithms were similar, corroborating the FDA findings, but with some differences in the modeled slopes and magnitude of changes. The alternative software resulted in an average 25% reduction in the 95% confidence intervals of the mixed effects models with generally lower Akaike Information Criterion values.

WHAT IS THE CURRENT KNOWLEDGE ON THE TOPIC?

☑ Evaluation of T‐wave morphology measures (J‐Tpeak and Tpeak‐Tend) are under regulatory consideration for ECG studies and may impact cardiac safety assessment of future medications beyond QTc interval.

WHAT QUESTION DID THIS STUDY ADDRESS?

☑ To date, no data exist on the reproducibility of these measurements using alternative software or methodology.

WHAT THIS STUDY ADDS TO OUR KNOWLEDGE

☑ This study corroborates findings reported by the FDA using a fully‐automated ECG analysis algorithm. It also highlights potential differences in interpretation, while providing more consistency in measuring drug effects with 25% reduction in confidence intervals.

HOW THIS MIGHT CHANGE CLINICAL PHARMACOLOGY OR TRANSLATIONAL SCIENCE

☑ Changes in the technical approach used to measure repolarization features have the potential to improve sensitivity of J‐Tp biomarker and reduce variability.

In 2005, the International Committee for Harmonization issued the E14 guidance requiring a thorough QT/QTc interval evaluation (TQT) for all drugs, with some exceptions, prior to regulatory approval (ICH E14).1 Over the last 12 years, this guidance has prevented drugs that increase the risk of fatal arrhythmia, torsade de pointes ventricular tachycardia, from reaching the marketplace. However, it is now recognized that potential false‐positive QTc findings associated with this regulatory endpoint may discourage development of other beneficial and safe medications.2 Therefore, new endpoints are being examined for regulatory consideration.

Recently the US Food and Drug Administration's (FDA) Critical Path Initiative funded their own investigation of alternative ECG biomarkers for detecting cardiac ion channel block using drugs known to cause QT prolongation and associated with varying incidences of torsade risk.3 The four drugs prospectively studied in a randomized, crossover design inhibit the human ether‐a‐go‐go‐related gene (hERG) potassium channel, IKr, either alone or in combination with varying degrees of inhibition on the L‐type calcium, and early and late sodium inward currents. Dofetilide was chosen as a specific hERG blocker with a high incidence of torsade.4, 5 Quinidine was also chosen as a strong hERG blocker at low concentrations but with calcium and sodium current inhibition at higher plasma concentrations.4 The last two drugs, ranolazine and verapamil, are associated with a low risk of torsade.6, 7 Although both are potent hERG blockers, ranolazine additionally inhibits the late sodium current, while verapamil inhibits the L‐type calcium current.4 Using only supine resting ECGs, it was shown that, unlike QTc, early and late measures of repolarization (heart rate‐corrected global J‐Tpeak (c) and global Tpeak–Tend, respectively) could differentiate pure hERG block associated with a high incidence of torsade risk, from multichannel block of hERG and either calcium or late sodium currents, which is associated with lower risk.3

To further advance the initiative to find improved ECG biomarkers, the 10‐sec ECGs extracted prespecified timepoints and the 24‐h continuously collected digital ECG data from the FDA study were archived at the University of Rochester Medical Center Telemetric and Holter ECG Warehouse (THEW) and made available for analyses by other investigators. Our objective was to remeasure the FDA QTcF and T‐wave morphology biomarker findings (J‐Tpeakc and Tpeak–Tend), recorded during supine rest at multiple prespecified timepoints using an alternative automated software (Rhythm Express, VivaQuant, St. Paul, MN), and to examine differences in results between the FDA and the alternative software.

RESULTS

Baseline values

Baseline demographics and vital signs of the subjects of this study were previously reported.3 The baseline ECG values determined in this study by the FDA and alternative software algorithms were, respectively: heart rate 56.0 ± 6.7 vs. 56.0 ± 6.6 bpm; QTcF 395.6 ± 17 vs. 394.1 ± 16.4 ms; J‐Tpeakc 224.1 ± 19.8 vs. 227.6 ± 21.2 ms; Tpeak–Tend 74.3 ± 6.9 vs. 77.6 ± 5.8 ms.

Pharmacokinetic analysis

Figure 1 shows the results of pharmacokinetic analysis for each drug: dofetilide, quinidine, ranolazine, and verapamil. The pharmacokinetic characteristics were previously reported.3 Dofetilide and quinidine had maximum concentrations occurring at 2.5 h and 2.0 h with mean half‐lives of 7.2 ± 1.1 (±SD) and 7.8 ± 1.5 h, respectively. Mean peak ranolazine concentrations occurred at 4.0 h with a half‐life of 7.5 ± 2.3 h and verapamil peak concentrations occurred at 1.0 h with a mean half‐life of 10.4 ± 3.2 h.

Figure 1

Measured plasma concentration (mean ± 95% CI) for dofetilide, quinidine, ranolazine, and verapamil.

QTcF, J‐Tpeakc, and Tpeak–Tend interval analyses

The differences between mean maximum (±95% confidence interval (CI)) effects of each drug on J‐Tpeakc were not statistically significant between the FDA and alternative software results (Figure 2). However, differences between the mean maximum (±95% CI) effects of dofetilide and quinidine on QTcF and Tpeak–Tend were statistically significant between the methods. The differences can be explained by 1) different approaches to detecting the peak of notched T‐waves and 2) the influence of U‐waves on T offset (end of T‐wave) detection. The FDA software consistently used the first peak of the notched T‐wave for the Tpeak annotation,8 while the alternative software was designed to use the last significant peak prior to downslope of the T‐wave. This resulted in shorter J‐Tpeakc and longer Tpeak–Tend durations (Figure 3 a) for the FDA measurements.

Figure 2

Comparison of placebo‐adjusted maximal (mean ± 95% CI) changes from baseline in QTcF, J‐Tpeakc, and Tpeak–Tend after dofetilide, quinidine, ranolazine, and verapamil.

Figure 3

Examples of differences between the FDA and Rhythm Express measurements of T‐wave offset (end of T‐wave). Only annotations of the alternative software are shown. The FDA and alternative interval values are shown below each recording. (a) The differences in the two methods in the presence of a notched T‐wave. (b) The influence of a U‐wave. [Color figure can be viewed at http://wileyonlinelibrary.com]

When a U‐wave was present, it appears that in some cases the FDA software included at least a portion of the U‐wave in the T‐offset, while the alternative software excluded the U‐wave entirely (Figure 3 b). The influence of these methodological differences is particularly evident in the case of dofetilide, for which the mean maximal change in QTcF was measured shorter by 15 ms by the alternative algorithm, mainly due to the shortening of the Tpeak–Tend interval (ΔΔTpeak–Tend = 39.72 ms for FDA and 20.33 ms for the alternative). The differences in software were also present for quinidine (e.g., ΔΔTpeak–Tend = 50.27 ms for the FDA measure vs. 26.12 ms for the alternative).

The consistency of measurements was evaluated per the ASTM standard.17 The triplicate measurements during a timepoint were treated as repeated measurements over a short period of time. The results are summarized in Supplementary Table S1. The average reduction in repeatability standard deviation between the FDA and Rhythm Express (RE) measurements is 16%. The highest improvement was achieved in measurements obtained on dofetilide and quinidine treatment for all metrics, but especially Tpeak–Tend. The alternative software was more consistent in determining placebo‐corrected changes from baseline, resulting in an average reduction in the CI of 25% across all models.

Exposure–response analysis

The comparison of the FDA and the alternative measures in the same exposure–response model for each drug is shown in Figure 4, with statistics presented in Table 1. As noted above, QT intervals tend to be shorter when measured by the alternative software, resulting in exposure–response slopes that are slightly smaller. CIs between the methods generally overlap. The differences between the slopes were tested with a two‐sample t‐test and were found not to be statistically significant, with the exception of QTcF for dofetilide (FDA slope = 28.64 vs. alternative slope = 23.74, P = 0.007), and quinidine (FDA slope = 42.36 vs. alternative slope = 33.19, P = 0.01), and Tpeak–Tend for dofetilide (FDA slope = 14.45 vs. alternative slope = 8.04, P = 0.001), and quinidine (FDA slope = 29.75 vs. alternative slope = 14.74, P = 0.005).

Figure 4

Comparison of drug‐induced changes (mean ± 95% CI) for the placebo‐corrected change from baseline from model predictions vs. plasma concentrations using either the FDA or Rhythm Express interval measurements. [Color figure can be viewed at http://wileyonlinelibrary.com]

Table 1

Summary of exposure–response analysis by two models: FDA intervals, a linear model based on reported FDA interval measurements; Rhythm Express (RE) intervals, a linear model based on Rhythm Express interval measurements

Dofetilide	FDA intervals			RE intervals
	QTcF	JTp	TpTE	QTcF	JTp	TpTE
Intercept	–3.9	0.7	–4.5	–2.0	1.3	–1.6
Slope	28.6	14.1	14.4	23.7	14.9	8.0
P‐value	6.7E‐54	1.2E‐18	4.0E‐16	6.4E‐54	4.0E‐29	1.5E‐10
95% Lower	25.6	11.2	11.1	21.3	12.5	5.7
95% Upper	31.6	17.1	17.8	26.2	17.2	10.4
AIC	2638	2478	2432	2489	2391	2037
CI reduction				17%	20%	28%
Quinidine	QTcF	JTp	TpTE	QTcF	JTp	TpTE
Intercept	3.3	5.6	‐2.1	1.9	3.9	0.6
Slope	42.4	11.6	29.8	33.2	16.4	14.7
P‐value	2.0E‐35	3.5E‐03	1.1E‐09	2.9E‐29	5.1E‐08	1.1E‐05
95% Lower	36.5	3.8	20.4	28.0	10.6	8.2
95% Upper	48.3	19.3	39.1	38.4	22.2	21.2
AIC	2587	2567	2685	2384	2454	2343
CI reduction				11%	25%	30%
Ranolazine	QTcF	JTp	TpTE	QTcF	JTp	TpTE
Intercept	2.1	2.4	0.2	1.0	–0.1	–0.5
Slope	4.4	–0.7	4.4	3.1	0.6	3.0
P‐value	1.3E‐07	3.8E‐01	3.3E‐07	1.7E‐09	4.0E‐01	2.3E‐06
95% Lower	2.8	–2.2	2.7	2.1	–0.8	1.8
95% Upper	6.0	0.8	6.1	4.1	2.1	4.3
AIC	2342	2222	2078	2249	2265	1843
CI reduction				39%	4%	26%
Verapamil	QTcF	JTp	TpTE	QTcF	JTp	TpTE
Intercept	2.6	–0.9	2.3	2.6	1.4	1.6
Slope	0.0	0.0	0.0	0.0	0.0	0.0
P‐value	4.5E‐01	8.7E‐01	1.5E‐02	1.2E‐01	5.6E‐03	7.5E‐01
95% Lower	0.0	0.0	0.0	–0.1	0.0	0.0
95% Upper	0.0	0.0	0.0	0.0	0.0	0.0
AIC	2276	2156	1898	2272	2191	1707

*P < 0.05 FDA model vs. RE model slopes.

Since it appears that a few subjects with complex T‐waves (either notched T and/or the presence of U‐waves) can create differences in the results and potentially change the interpretation of the data, we further examined scatterplots of all points for both the FDA and the alternative methods. Figures 5 and 6 show exposure–response scatterplots for ΔΔQTcF, J‐Tpeakc, and Tpeak–Tend values from both systems for dofetilide and quinidine treatments. The alternative software was more consistent in determining the fiducial endpoints, resulting in 11–30% smaller CIs for these two drugs in all metrics and a 39% reduction in CIs for QTcF for ranolazine. An overall 7% better goodness of fit (smaller Akaike Information Criterion (AIC)) was observed for the alternative software compared to the FDA algorithm for quinidine, dofetilide, and ranolazine. The effect of verapamil was insignificant, resulting in little difference in statistics of model fit between the two algorithms.

Figure 5

Scatterplots of placebo‐adjusted changes from baseline in QTcF, J‐Tpeakc, and Tpeak–Tend measurements from all subjects and timepoints after dofetilide using the FDA and Rhythm Express algorithms. [Color figure can be viewed at http://wileyonlinelibrary.com]

Figure 6

Scatterplots of placebo‐adjusted changes from baseline in QTcF, J‐Tpeakc, and Tpeak–Tend measurements from all subjects and timepoints after quinidine using the FDA and Rhythm Express algorithms. [Color figure can be viewed at http://wileyonlinelibrary.com]

DISCUSSION

In this study we retrospectively examined data from a randomized controlled clinical trial conducted by the FDA3 for potential new biomarkers that can differentiate multichannel block and improve assessment of arrhythmia risk compared to QTc alone. Our study objectives were to determine if the use of an alternative software algorithm could corroborate the FDA analysis and to examine the nature of differences in the results of the two algorithms.

The main finding in this study was corroboration of the FDA's observations3 on the ability of ECG biomarkers to identify single and multichannel block by drugs. However, the two methods did differ occasionally in how fiducial endpoints were determined in individual cases and, in some cases, the group means were affected modestly. This observation brings emphasis to the need to carefully evaluate the influence of algorithm methodological differences on results of ECG biomarkers. The alternative software algorithm consistently selected the last significant peak of the T‐wave prior to its downslope for the T‐peak annotation, while the FDA algorithm consistently selected the first peak.8 This resulted in longer J‐Tpeakc values assessed by the alternative algorithm (Table 1, Figure 2). Examination of the results for J‐Tpeakc across all drugs showed that there were no statistically significant differences in mean maximal values or slope of the exposure–response relationship. However, trends were apparent: the alternative software produced a steeper slope for J‐Tpeakc for quinidine (11.57 (FDA) vs. 16.40 (alternative)) and ranolazine (–0.66 (FDA) vs. 0.6 (alternative)).

In a subsequent study from the same group,9 J‐Tpeakc was shown to be a stronger predictor of risk than Tpeak–Tend. However, the consistency of J‐Tpeakc measurement can be undermined in the presence of notched T‐wave. It is therefore beneficial to measure J‐Tpeakc in a manner that maximizes its sensitivity and improves consistency. The comparison of exposure–response analysis results demonstrated that selecting the last significant peak of the T‐wave prior to its downslope results in a higher concentration slope, which increased sensitivity. This effect is also complemented with reduction of CI, which should theoretically increase specificity of the biomarker. We hypothesize that using the first peak to measure J‐Tpeakc, as was done by the FDA algorithm,8 has the potential to bias a drug toward being more safe (false negative), given the hypothesis that a shorter J‐Tpeakc interval may offset the risk associated with QT prolongation.3 In the absence of a gold standard, however, it is not possible to know which result is more correct without long‐term follow‐up in a large cohort of patients.

Contrary to the J‐Tpeakc results, the presence of outliers among the FDA‐measured Tpeak–Tend values resulted in a longer average. Analysis of waveforms and annotations in outlier cases (Figure 3) indicated that the FDA algorithm may estimate a longer Tpeak–Tend for two reasons. First, selecting the first peak on a notched T‐wave as the T‐peak results in longer Tpeak–Tend. Second, T‐offset detection by the FDA algorithm seemed to be biased by the presence of U‐waves, especially when the T‐wave was flat (Figure 3 b). The same bias also resulted in longer QTcF values. The alternative software consistently excluded the U‐wave from offset of the T‐wave.

While this study corroborates the FDA methods and findings, the observation of differences between the two methods is also helpful. Although there is no gold standard by which the correctness of findings by one or the other algorithm can be judged, the consistency of the interval readings is important. Reductions in variability can improve confidence in study interpretation, increase specificity (i.e., reduce false positives), reduce the sample study size required for endpoint determination, and potentially reduce cost. While it is possible that the smaller variance in intervals measured with the alternative technique might have been due to measurement error, the analysis of the repeatability standard deviation within the 5‐min timepoints suggests that improved consistency of measurements contributed to the reduction in CI. ECGs recorded in individual resting subjects during a 5‐min interval would be expected to be nearly identical. Thus, variation of those values is a more precise test of measurement repeatability than the assessments done on the whole population over all timepoints, because those assessments include intersubject variability and drug‐ and circadian‐related intertimepoint variability, in addition to measurement variability.

It is noteworthy that the alternative software provided fully automated continuous beat‐to‐beat interval measurements for all timepoints. No editing was performed on the results provided by the alternative software. The analysis in this report was based on 10‐sec ECG data extracted at prespecified timepoints from the continuous data stream. Analysis of the continuous data would allow for capture of varying autonomic states such as eating, sleep, and ambulation. This capability could be used to investigate the utility of temporally dynamic parameters such as QT beat‐to‐beat10 and ECG restitution11 and to compare them against the FDA T‐wave morphology parameters. This could optimize the utility of potential biomarkers of arrhythmia risk.

In summary, we have corroborated the FDA's discovery that simple ECG biomarkers can be used to differentiate ion channel block by a drug, even when multiple channels are blocked. In addition, we have shown that technical differences in the measurement methodologies can result in clinically meaningful differences in the results, both at the level of drug assessment and of individual patient/subject assessment. Careful attention to the influence of algorithm variations is warranted. Without a gold standard for guidance, it would be reasonable to seek algorithms that limit the presence of outliers and the resultant variability.

METHODS

All ECGs were obtained from the E‐OTH‐12‐5232‐020 (FDA‐1) database archived in the Telemetric and Holter ECG Warehouse at the University of Rochester Medical Center, Rochester, NY. The downloaded 5,232 10‐sec ECG segments were processed by Rhythm Express software to generate the vector magnitude ECG of the vectorcardiogram12 and automatically measure cardiac intervals. The intervals were averaged for each 10‐sec segment. J‐Tpeak and QT were corrected for heart rate.3 The FDA annotations for each 10‐sec measurement were downloaded from Physionet13 and were used to confirm the statistical models and compute additional exposure–response parameters. Averaged interval results and summary statistics were reviewed and select segments with outlier averages or large differences from the FDA results were reviewed for accuracy without making any changes to the automated interval measurements.

Study design

The archived ECG data were from a randomized controlled five‐way single‐dose crossover clinical trial in 22 healthy volunteers (11 females) conducted at a phase I clinical research unit (Spaulding Clinical, West Bend, WI). The study details and inclusion criteria were previously reported3, 8 and approved by the US Food and Drug Administration Research Involving Human Subjects Committee and the local Institutional Review Board. All subjects gave written informed consent.

As per the previous description,3 in the morning of the treatment period the subjects received one of four drugs or placebo under fasting conditions. There was a 7‐day washout period between each 24‐h treatment period, so subjects received treatments on days 1, 9, 17, 25, and 33. Prior to dosing, a continuous 12‐lead ECG recorder (Surveyor, Mortara Instrument, Milwaukee, WI) using the Mason‐Likar electrode14 configuration was connected to each subject. The continuous ECG recordings were made at 500 Hz and with an amplitude resolution of 2.5 μV. From the continuous recording, three replicate 10‐sec ECGs (pre ‐and postdose) were extracted at 16 predefined timepoints (predose and 0.5, 1, 1.5, 2, 2.5, 3, 3.5, 4, 5, 6, 7, 8, 12, 14, 24 h postdose), during which the subjects were resting in a supine position for 10 min. After each ECG extraction timepoint period, a blood sample was drawn for pharmacokinetic analysis. Plasma drug concentration was determined using a validated liquid chromatography with tandem mass spectroscopy method by Frontage Laboratories (Exton, Philadelphia, PA).

ECG assessment

Rhythm Express software was used to perform automated measurement of cardiac intervals. The software removes of up to 95% of noise, without distorting ECG morphology, and performs arrhythmia screening and beat‐to‐beat cardiac interval measurement using wavelet‐based techniques.15, 16 Visualization tools facilitate review of the results with synchronized display of ECGs, fiducial point markers, and interval trends, along with automatically detected interval outliers and abnormal beats. The wavelet‐based emphasis signal and its derivative are used to identify T‐wave morphology and facilitate peak and offset search. In case of a notched T‐wave, T peak is detected as the last significant peak prior to T‐wave downslope. If multiple T‐wave peaks are present, the temporal relationship of peaks with amplitudes greater than three‐quarters of the maximum T‐wave amplitude are evaluated in relationship to the peaks of the emphasis signal to identify the last peak prior to T‐wave downslope. In case of complex T‐wave morphology, the search window for T peak starts after the point of the maximum change, identified as the first extremum (maximum or minimum) of the emphasis signal derivative. Subsequent positive and negative peaks in the emphasis signals are compared to identify T‐wave morphology and find the last significant peak. The wavelet‐based emphasis signal enhances the significant peaks and smoothes out noise‐related jitter in ECG interval values. The T‐offset detection algorithm uses a combination of ECG, derivative, and emphasis signal threshold methods, depending on the identified T‐wave morphology characteristics and measured noise level.

Statistical analysis

The placebo‐corrected change from baseline was computed using lme4 and lsmeans packages in R 3.3.1 (R Foundation for Statistical Computing, Vienna, Austria). The change from baseline for each ECG biomarker (ΔQTcF, ΔTpeak–Tend and ΔJ‐Tpeakc) by timepoint was the dependent variable, for which baseline was defined as the average predose value. Period, time, treatment, and an interaction between treatment and time were included as fixed effects, and subject was included as a random effect. Exposure–response analysis was performed with a linear mixed‐effects model, fitted using lme4 and lsmeans, with the placebo‐corrected change from baseline as dependent variable and fixed effects on the intercept and corresponding time matched concentration and a random intercept and slope per subject. The FDA annotations were fitted in the same statistical models and the resulting values were compared to partial published results to confirm the models.

Differences between mean maximum drug effects and slopes for each method were compared using a two‐sample t‐test. P < 0.05 was considered statistically significant.

AUTHOR CONTRIBUTIONS

A.F., M.B., and J.M. wrote the article; J.M. designed the research; M.B., J.M., and A.F. performed the research; M.B. analyzed the data; M.B. contributed new analytical tools.

CONFLICT OF INTEREST/DISCLOSURE

Marina Brockway is CTO and shareholder of VivaQuant LLC. Anthony A. Fossa is a shareholder in VivaQuant LLC.

Table S1. Repeatability standard deviations

Click here for additional data file.