Data-driven drug-induced QT prolongation surveillance using adverse reaction signals derived from 12-lead and continuous electrocardiogram data.

Drug-induced QT prolongation is one of the most common side effects of drug use and can cause fatal outcomes such as sudden cardiac arrest. This study adopts the data-driven approach to assess the QT prolongation risk of all the frequently used drugs in a tertiary teaching hospital using both standard 12-lead ECGs and intensive care unit (ICU) continuous ECGs. We used the standard 12-lead ECG results (n = 1,040,752) measured in the hospital during 1994-2019 and the continuous ECG results (n = 4,835) extracted from the ICU's patient-monitoring devices during 2016-2019. Based on the drug prescription frequency, 167 drugs were analyzed using 12-lead ECG data under the case-control study design and 60 using continuous ECG data under the retrospective cohort study design. Whereas the case-control study yielded the odds ratio, the cohort study generated the hazard ratio for each candidate drug. Further, we observed the possibility of inducing QT prolongation in 38 drugs in the 12-lead ECG analysis and 7 drugs in the continuous ECG analysis. The seven drugs (vasopressin, vecuronium, midazolam, levetiracetam, ipratropium bromide, nifedipine, and chlorpheniramine) that showed a significantly higher risk of QT prolongation in the continuous ECG analysis were also identified in the 12-lead ECG data analysis. The use of two different ECG sources enabled us to confidently assess drug-induced QT prolongation risk in clinical practice. In this study, seven drugs showed QT prolongation risk in both study designs.

Introduction

The prolongation of the QT interval refers to the extension of the interval between the start of the QRS complex and the end of the T wave by external factors. The delay in ventricular repolarization caused by a reduction in the outward potassium current results in the broadening of ventricular action potentials and, consequently, the prolongation of the QT interval [1]. QT prolongation may cause diverse arrhythmic conditions, including torsade de pointes, which is a type of ventricular tachycardia known to cause sudden cardiac death [2-4].

Drug-induced QT prolongation is the most common cause of acquired QT prolongation [5,6]. Furthermore, A drug’s propensity to cause QT interval prolongation may cause its withdrawal from the market [7]. Accordingly, the early detection of drug-induced QT prolongation is crucial from the medical and socioeconomic perspectives.

Many studies have examined drug-induced QT prolongation, but most of these investigations have the following limitations. In many cases, researchers first select the drug to be investigated [5,8]. This approach enables researchers to investigate the risks for only some drugs of interest. Hence, such risk assessment studies often exclude drugs that are known to be less lethal, such as those used in chronic disease or symptomatic treatments. As the risk of QT prolongation is higher in patients with chronic disease than in the general population [9,10], comprehensive studies on the QT prolongation risks of the relevant less lethal drugs are required. To reduce this selection bias, we selected hundreds of candidate drugs, including those that had not been studied before, using a data-driven approach according to the drugs’ frequency of use in a tertiary teaching hospital.

The second limitation of existing studies is that they are based on a standard 12-lead ECG alone. The measuring time for standard 12-lead ECGs is only 10 seconds at a time, and there are inconsistent days to years of time gaps between the two measurements. Since a patient’s drug adherence for the period between two measurements is unknown, it is difficult to capture the exact time gap between drug exposure and adverse drug events and identify acute adverse drug events. This causes researchers to hesitate in performing retrospective cohort studies with standard 12-lead ECG data.

Therefore, in addition to using standard 12-lead ECG data, this study used the continuous ECG data extracted from an intensive care unit (ICU) patient-monitoring device. As the administration of all drugs was recorded in the ICU, we accurately identified the exact time point of drug exposure and the time until the QT prolongation event. Further, using continuous ECG data, we conducted a retrospective cohort study, which is considered to obtain higher levels of evidence than case-control studies, and investigated acute adverse drug events. However, it has limited generalizability because the continuous ECG was usually measured in an intensive care unit (ICU), and the characteristics of patients and popularly used drugs are different between ICU and general wards.

To make up for the limitation of standard 12-lead ECG data and the continuous ECG data extracted from an ICU patient-monitoring device, we used both data together in evaluating the QT prolongation risk of drugs. We built two separate algorithms to analyze the risk of QT interval prolongation by adopting the data-driven approach and conducted a case-control study using standard 12-lead ECG data and a retrospective cohort study using continuous ECG data.

Materials and methods

The Institutional Review Board of Ajou University Hospital approved this study (IRB No. AJIRB-MED-MDB-19-406) and waived the requirement for informed consent because the study retrospectively used anonymized data.

Dataset

We used two ECG sources. The first comprised standard 12-lead ECG data [11]. We extracted the standard 12-lead ECG data from the ECG MUSE system (GE Healthcare) of Ajou University Hospital for the period between 1994 and 2018 and linked them to electronic health record (EHR) data. Accordingly, we linked the 1,040,752 12-lead ECG data to the diagnosis, prescription, and procedure records of 447,632 patients. Since we extracted data regardless of visit or admission type, all patients (outpatients, inpatients admitted to general wards/ICUs, patients visited emergence department, etc.) were included.

The second ECG source included continuous ECG data. We extracted continuously monitored ECG data from the patient-monitoring devices (Philips [Amsterdam, The Netherlands] and Nihon Kohden [Tokyo, Japan]) of 4,835 patients in the hospital’s ICU for the period between 2016 and 2018. Further, we calculated the QT interval from the raw signal using the method proposed by Chesnokov algorithm et al. [12], corrected it using the Bazett formula [13] to obtain the QTc value, and recorded the median QTc interval for every 10 seconds. Finally, we linked the data to EHR for analysis.

As a reference for drugs known to prolong QT interval, we used the QT risk drug list provided by CredibleMeds [14], since this list is commonly used as a reliable reference in QT prolongation studies [15,16]. The list classifies drugs into four categories: the drugs that should be avoided in treating congenital long QT syndrome, those with a known risk of QT prolongation, those with a possible risk of QT prolongation, and those with a conditional risk of QT prolongation.

Case-control analysis using 12-lead ECG data

Candidate drug selection

To create the list of candidate drugs for analysis, we extracted all ECG results with QT prolongation (QTc>450 for men and QTc>460 for women) and randomly selected one ECG with several prolonged QT intervals for a single patient [17-19]. Further, we extracted all prescription data seven days before the date of ECG measurements in the QT prolongation cases, counting the frequency of each drug use. We also included all drugs to evaluate the QT prolongation risk of all drugs, including those not unknown yet. However, drugs prescribed more than 500 times in the cases were included to secure the statistical power. To reduce indication bias, we excluded the prescriptions ordered on the ECG measurement dates. Finally, based on the QT risk drug list provided by CredibleMeds [14], the candidate drugs were divided into subgroups with four levels by a clinician: Rank 4, 3, 2, and 1 drugs have a known risk, a possible risk, a conditional risk, and an unknown risk of QT prolongation, respectively.

Study design and population

As shown in Fig 1, we randomly selected one ECG result per patient (n = 447,632). After excluding the ECG results without gender or age information or with age outliers (n = 3,050), we included 444,582 ECG results in the study. Further, we adopted the propensity score matching method to match the control group (subjects whose QTc interval was within normal range) with the case group (subjects whose QTc interval was prolonged) to adjust confounding variables with the following covariates: Gender and age at the ECG examination date, the latest serum potassium and calcium levels calculated within a year of the ECG measurement date, the comorbidities recorded in the EHR within a year of the ECG measurement date (e.g., myocardial infarction, congestive heart failure, ischemic stroke, hemorrhagic stroke, diabetes mellitus, hypothyroidism, renal disease, AIDS/HIV, alcohol abuse, drug abuse, liver disease, and severe liver disease), and the frequency of drug use for each drug rank group within seven days of the ECG date. S1 Table provides the complete drug list of each drug rank that was used in counting the frequency. For patients without laboratory test results dated within one year of the ECG examination date, we replaced the missing values with the median values of patients of the same age group divided by 10-year intervals.

Fig 1

Flow chart of the research process.

Retrospective cohort analysis using continuous ECG data

We performed a survival analysis based on a retrospective cohort study using continuous ECG data to identify drugs having QT prolongation risk. We excluded all patients with records indicating ICU hospitalization prior to the study period (n = 311) from the analysis. Similarly, we excluded all patients who had ECG monitoring data with durations less than 24 hours or more than 30 days (n = 1,794). We excluded all patients under 15 years of age (n = 379), as well.

To secure at least 5% of the exposure group for further analysis, we selected 78 drugs prescribed for more than 5% of the study subjects as candidate drugs. Eighteen drugs were already classified as QT prolongation risk drugs by CredibleMeds; these are used for propensity score matching (see “Matching exposure and non-exposure groups”). Therefore, we finally analyzed the remaining 60 drugs.

Definition of index time and other variables

We defined the first drug initiation time of a target drug for each patient as the drug’s index time. We observed the QT intervals for 12 hours from the index time to assess the QT prolongation risk. For each covariate, all the drug infusions within 12 hours of the index time and all medical diagnosis records made before the index time were extracted from EHR data. The laboratory results of calcium and potassium levels before and after seven days of the index time were also extracted.

We defined QT prolongation as QTc>450 in men and QTc>460 in women [17-19]. Further, we excluded all patients who showed QT prolongation within 5 minutes before the index time. Finally, we measured the duration from the index time to the time of occurrence of QT prolongation for the survival analysis.

Matching exposure and non-exposure groups

We matched the drug-exposed and unexposed groups (or exposure and non-exposure groups) to eliminate the influence of confounding variables. Since a patient’s condition may vary during his or her hospital stay, we first sliced the continuous ECG data of the non-exposure group into 12-hour-long segments and selected the segment having the closest length-of-stay to the length-of-stay at the index time of the exposure group.

To match the groups, we applied the propensity score matching method with the following covariates: gender, age, ward type, calcium and potassium levels (closest laboratory record with the index time), 18 for known QT prolongation risk [14] and used in more than 5% of ICU patients, and 9 QT-related clinical factors of more than 1% of ICU patients. Each patient in the exposure group was matched to four patients in the non-exposure group. S2 Table shows the complete list of drugs and comorbidities used in propensity score matching.

Statistical analysis

We first compared subjects’ demographic characteristics (gender and age), laboratory test results (calcium and potassium levels), and comorbidities among the two study designs using Pearson’s chi-square test (for categorical data) and the independent two-sample t-test (for continuous data). Further, for the case-control analysis based on 12-lead ECG data, we performed conditional logistic regressions for each drug in the candidate drug list. Subsequently, we estimated the odds ratios (ORs) and 95% confidence intervals (CIs) of each drug with a significance level of p < 0.05. For the retrospective cohort analysis based on continuous ECG data, we established the Cox Proportional-Hazards Model to calculate the hazard ratios (HRs) of QT prolongation for each drug and 95% CIs of each drug at a significance level of p < 0.05.

Further, to validate our methods, we analyzed the QT prolongation risk of each drug in the drug list provided by CredibleMeds as the positive control. While analyzing the drugs in the QT drug list, we excluded the drug from the matching variable in propensity score matching and calculated the variable’s ORs or HRs.

To correct the multiple comparison problem, we calculate the false discovery rate and validated the results at a significance level of p < 0.05. Data management was performed using Azure data studio version 1.19.0, and all statistical analyses were conducted using R version 4.0.2.

Results

Baseline characteristics

Table 1 depicts the study subjects’ baseline characteristics prior to propensity score matching for the case-control analysis and the subjects’ characteristics for the retrospective cohort analysis. In the case-control analysis, the mean age of subjects was higher in the case group than in the control group, and the laboratory test results (calcium and potassium levels) were higher in the control group than in the case group. Further, comorbidities, except hypothyroidism and AIDS, were higher in the case than in the control group. After propensity score matching, 58,505 QT prolongation cases and their matched 58,505 controls were enrolled in the analysis for the case-control analysis.

Table 1

Baseline characteristics of subjects.

	QT prolongation
	Case-control analysis			Retrospective cohort analysis
	Control group	Case group	p-value	No. of QT prolongation < 1	No. of QT prolongation ≥ 1	p-value
N	386,077	58,505		1,371	980
Gender, male, n (%)	189,146 (49.0)	29,856 (51.0)	<0.001	997 (72.7)	635 (64.8)	<0.001
Age, mean (SD)	42.4 (20.4)	55.1 (20.8)	<0.001	52.3 (20.6)	59.7 (18.8)	<0.001
Potassium, mean (SD)	4.1 (0.4)	4.0 (0.6)	<0.001	3.9 (0.8)	3.9 (1.0)	0.251
Calcium, mean (SD)	9.2 (0.6)	8.8 (0.8)	<0.001	8.0 (1.6)	7.9 (1.7)	0.086
Myocardial infarction, n (%)	3,457 (0.9)	1,723 (2.9)	<0.001	60 (4.4)	54 (5.5)	0.244
Congestive heart failure, n (%)	2,390 (0.6)	1,812 (3.1)	<0.001	15 (1.1)	33 (3.4)	<0.001
Ischemic stroke, n (%)	6,332 (1.6)	2,519 (4.3)	<0.001	41 (3.0)	73 (7.4)	<0.001
Hemorrhagic stroke, n (%)	2,207 (0.6)	1,480 (2.5)	<0.001	279 (20.4)	142 (14.5)	<0.001
Diabetes mellitus, n (%)	12,364 (3.2)	3,284 (5.6)	<0.001	40 (2.9)	53 (5.4)	0.003
Renal disease, n (%)	3,074 (0.8)	2,068 (3.5)	<0.001	31 (2.3)	41 (4.2)	0.011
Hypothyroidism, n (%)	1,999 (0.5)	255 (0.4)	0.010	NA	NA	NA
AIDS/HIV, n (%)	123 (0.0)	22 (0.0)	0.552	NA	NA	NA
Alcohol abuse, n (%)	1,145 (0.3)	921 (1.6)	<0.001	NA	NA	NA
Drug abuse, n (%)	392 (0.1)	199 (0.3)	<0.001	NA	NA	NA
Liver disease, n (%)	1,717 (0.4)	1,350 (2.3)	<0.001	NA	NA	NA
Severe liver disease, n (%)	282 (0.1)	424 (0.7)	<0.001	NA	NA	NA
Sepsis, n (%)	NA	NA	NA	16 (1.2)	23 (2.3)	0.041
Sudden cardiac arrest, n (%)	NA	NA	NA	16 (1.2)	51 (5.2)	<0.001
AV block, n (%)	NA	NA	NA	19(1.4)	17(1.7)	0.611

AV block, atrioventricular block; NA, not applicable; SD, standard deviation.

In the retrospective cohort analysis, we finally selected 2,351 patients for the analysis, and according to the target, those patients were subdivided into the non-exposure group and the exposure group. Table 1 shows the baseline characteristics of subgroups according to whether QT prolongation was observed at least once or not. The age, incidences of diabetes, sudden cardiac arrest, sepsis, and congestive heart failure were higher in the patients who ever have experienced QT prolongation at least one time.

Statistical analysis results

Case-control analysis

The drug selection process identified the following candidate drugs: 167 rank 1, 15 rank 2, 8 rank 3, and 14 rank 4 drugs. 64.29% of the rank 4 drugs showed a QT prolongation risk with a significance level p < 0.05; this is the highest percentage among all ranking groups. As shown in Fig 2, the percentages of drugs with QT prolongation risks in each ranking group were consistent with the order of QT risk levels indicated by CredibleMeds.

Fig 2

Results of the positive rates of each drug group calculated to validate the algorithm.

As shown in Table 2, 38 rank 1 drugs (i.e., drugs with unknown risk of QT prolongation) showed significant QT prolongation risk at the p < 0.05 significance level. The five drugs with the highest risks of QT prolongation were an antidiuretic hormone (OR [95% CI], 2.05 [1.97–2.13]); somatostatin, a growth hormone–inhibiting hormone (OR [95% CI], 1.94 [1.77–2.11]); etomidate, a short-acting intravenous anesthetic agent (OR [95% CI], 1.81 [1.77–1.85]); methylergometrine, a smooth muscle constrictor (OR [95% CI], 1.8 [1.68–1.92]); and lorazepam, a benzodiazepine acting on the brain and nerves (OR [95% CI], 1.79 [1.74–1.84]). S3 Table depicts the complete results for all ranking groups.

Table 2

Results of 38 rank 1 drugs with significant QT prolongation risks.

Drug	OR	CI (95%)	p-value
Vasopressin	2.05	1.97–2.13	<0.001
Somatostatin	1.94	1.77–2.11	<0.001
Etomidate	1.81	1.77–1.85	<0.001
Methylergometrine	1.80	1.68–1.92	<0.001
Lorazepam	1.79	1.74–1.84	<0.001
Vecuronium Bromide	1.68	1.63–1.73	<0.001
Hydrocortisone	1.67	1.56–1.78	<0.001
Ceftriaxone	1.65	1.61–1.69	<0.001
Ipratropium Bromide	1.64	1.6–1.68	<0.001
Levetiracetam	1.64	1.58–1.7	<0.001
Perindopril	1.55	1.48–1.62	<0.001
Labetalol	1.55	1.49–1.61	<0.001
Ceftazidime	1.52	1.42–1.62	<0.001
Rosuvastatin	1.35	1.31–1.39	<0.001
Carvedilol	1.34	1.29–1.39	<0.001
Morphine	1.32	1.26–1.38	<0.001
Spironolactone	1.31	1.25–1.37	<0.001
Chlorpheniramine	1.31	1.29–1.33	<0.001
Isosorbide Dinitrate	1.30	1.27–1.33	<0.001
Clopidogrel	1.25	1.22–1.28	<0.001
Remifentanil	1.18	1.13–1.23	<0.001
Midazolam	1.18	1.09–1.27	<0.001
Propacetamol	1.17	1.09–1.23	<0.001
Ibuprofen	1.24	1.17–1.31	0.0028
Ramipril	1.17	1.11–1.23	0.0028
Hydralazine	1.49	1.36–1.62	0.003
Captopril	1.31	1.21–1.41	0.0062
Levocloperastine	1.25	1.17–1.33	0.0065
Ticagrelor	1.32	1.22–1.42	0.0093
Clindamycin	1.16	1.05–1.27	0.0095
Theobromine	1.23	1.15–1.31	0.0124
Nifedipine	1.17	1.03–1.31	0.0133
Cefotetan	1.11	1.03–1.19	0.0233
Valproate	1.14	1.01–1.27	0.0453
Tiotropium	1.31	1.17–1.45	0.0455
Propranolol	1.18	1.05–1.31	0.0458
Erdosteine	1.25	1.13–1.37	0.0486
Cefpiramide	1.12	1.05–1.19	0.0487

CI, confidence interval; OR, odds ratio.

Retrospective cohort analysis

Vasopressin (HR [95% CI], 1.49 [1.33–1.65]), vecuronium (1.76 [1.53–1.99]), midazolam (1.76 [1.53–1.46]), levetiracetam (1.43 [1.25–1.61]), ipratropium bromide (1.4 [1.32–1.48]), nifedipine (1.33 [1.16–1.5]), and chlorpheniramine (1.06 [1.02–1.1]) showed significant QT prolongation risks at p < 0.05 (Table 3). These seven drugs revealed significant QT prolongation risks in the case-control study, as well. Among 18 drugs on the drug list provided by CredibleMeds, 12 (66%) showed significant QT prolongation risk. S4 Table depicts the complete analysis results.

Table 3

Hazard and odds ratios for seven drugs with significant QT prolongation risk.

	Retrospective study			Case-control study
	HR	CI (95%)	p-value	OR	CI (95%)	p-value
Vasopressin	1.49	1.33–1.65	0.024	2.05	1.97–2.13	<0.001
Vecuronium	1.76	1.53–1.99	0.021	1.68	1.63–1.73	<0.001
Midazolam	1.37	1.27–1.47	0.028	1.18	1.16–1.21	<0.001
Levetiracetam	1.43	1.25–1.61	<0.001	1.51	1.3–1.72	<0.001
Ipratropium bromide	1.4	1.32–1.48	<0.001	1.64	1.60–1.68	<0.001
Nifedipine	1.33	1.16–1.5	0.008	1.17	1.03–1.31	0.005
Chlorpheniramine	1.06	1.02–1.1	0.008	1.31	1.29–1.33	<0.001

HR, hazard ratio; OR, odds ratio.

Discussion

This study adopted a data-driven approach and used two different ECG sources to analyze the QT prolongation potential of 167 drugs using standard 12-lead ECG data and 60 drugs using continuous ECG data. It revealed the possibility of inducing QT prolongation in 38 drugs in the standard 12-lead ECG analysis and 7 in the ICU continuous ECG analysis.

QT prolongation is one of the most well-known side effects of drug use [5,6,20,21], and numerous studies have been conducted on this aspect [22]. Nevertheless, studies on the possibility of prolonging QT side effects in clinical practice remain insufficient because such studies generally focus on only a limited number of drugs selected by clinicians. Since clinicians are primarily interested in only a few drugs associated with the diseases treated by them, they may not consider drugs without specific indications. To reduce this bias, we selected candidate drugs based only on the number of prescriptions in our study. Therefore, we could observe the possibility of inducing QT prolongation even in drugs prescribed for conservative treatment, such as vecuronium and naproxen.

Many earlier studies have the limitation that they only used 12-lead ECG data for analysis [14,23]. It is difficult to identify acute adverse drug effects in standard 12-lead ECG data analysis due to the short ECG measurement time and the large time gap between drug administration and ECG measurement. In this study, continuous ECG data were extracted from ICU patient-monitoring devices during the period from hospitalization to discharge. Hence, we could analyze ECG data both before and after drug administration. Further, by using continuous ECG data, we identified the acute adverse drug effects that could occur within 12 hours of drug initiation. An ICU patient’s hospital stay is as short as 3–7 days; however, ICU patients are highly likely to suffer severe conditions and fatal complications, such as sudden cardiac arrest following QT prolongation [24]. Therefore, it is essential to investigate the occurrence of acute adverse drug effects during patients’ ICU stay.

This study has the following limitations: First, the detection of QT prolongation risk can be confounded by drug–drug interaction (DDI). In particular, DDI can be a major issue in the ICU [25]. Second, the study used a database comprising data that were retrospectively collected within a single institution. Future studies should perform multicenter and multinational research to obtain more comprehensive results. Third, the study did not account for indication bias. Indication bias refers to the case where QT prolongation occurs when drugs are prescribed to treat a specific condition that may cause QT prolongation, even though the drugs did not have any specific QT prolongation effect. Lastly, not all drugs used in the subject hospital were analyzed. It was to secure the least number of patients exposed to the target drug. If the drugs are more used, those drugs will be able to be included in our analysis in the future.

Conclusions

In this study, we analyzed the possibilities of QT interval prolongation of drugs by adopting a data-driven approach and using two large ECG sources, standard 12-lead ECG data (n = 444,582) and continuous ECG data (n = 2452). Consequently, we observed QT interval prolongation risk in 38 out of 167 drugs with unknown risk in the candidate drug list in a case-control study based on a standard 12-lead ECG database and 7 out of 60 drugs in a retrospective cohort study based on a continuous ECG database.

Complete drug list for each drug rank used in counting the frequency of drug use for each drug ranking group within seven days before the ECG measurement date.

Rank 1 drugs are candidate drugs with unknown QT prolongation risk in the case-control study. The drugs in other ranks were analyzed to validate the study.

(DOCX)

Click here for additional data file.

Complete list of the comorbidities and drugs used in the propensity score matching process in survival analysis.

The QT drug list is based on the QT risk drug list provided in CredibleMeds.org.

(DOCX)

Click here for additional data file.

Complete results for all drugs in each of the four drug ranks.

The candidate drug list is the list of drugs in rank 1; the drugs in other ranks were analyzed to validate the study. The drugs in each rank were used to count the drug use frequency of each rank.

(DOCX)

Click here for additional data file.

Complete analysis results of the 78 drugs considered in the survival analysis based on the continuous ECG database.

(DOCX)

Click here for additional data file.

6 Dec 2021

Data-driven drug-induced QT prolongation surveillance using adverse reaction signals derived from 12-lead and continuous electrocardiogram data

PLOS ONE

Dear Dr. Yoon,

Thank you for submitting your manuscript to PLOS ONE. After careful consideration, we feel that it has merit but does not fully meet PLOS ONE’s publication criteria as it currently stands. Therefore, we invite you to submit a revised version of the manuscript that addresses the points raised during the review process.

The paper is clearly written and the methods well described. We suggest to explain why the socalled credmeds durgs were excluded.

Please submit your revised manuscript by Jan 20 2022 11:59PM. If you will need more time than this to complete your revisions, please reply to this message or contact the journal office at plosone@plos.org. When you're ready to submit your revision, log on to https://www.editorialmanager.com/pone/ and select the 'Submissions Needing Revision' folder to locate your manuscript file.

Please include the following items when submitting your revised manuscript:

A rebuttal letter that responds to each point raised by the academic editor and reviewer(s). You should upload this letter as a separate file labeled 'Response to Reviewers'.

A marked-up copy of your manuscript that highlights changes made to the original version. You should upload this as a separate file labeled 'Revised Manuscript with Track Changes'.

An unmarked version of your revised paper without tracked changes. You should upload this as a separate file labeled 'Manuscript'.

If you would like to make changes to your financial disclosure, please include your updated statement in your cover letter. Guidelines for resubmitting your figure files are available below the reviewer comments at the end of this letter.

If applicable, we recommend that you deposit your laboratory protocols in protocols.io to enhance the reproducibility of your results. Protocols.io assigns your protocol its own identifier (DOI) so that it can be cited independently in the future. For instructions see: https://journals.plos.org/plosone/s/submission-guidelines#loc-laboratory-protocols. Additionally, PLOS ONE offers an option for publishing peer-reviewed Lab Protocol articles, which describe protocols hosted on protocols.io. Read more information on sharing protocols at https://plos.org/protocols?utm_medium=editorial-email&utm_source=authorletters&utm_campaign=protocols.

We look forward to receiving your revised manuscript.

Kind regards,

Chiara Lazzeri

Academic Editor

PLOS ONE

Journal Requirements:

When submitting your revision, we need you to address these additional requirements.

1. Please ensure that your manuscript meets PLOS ONE's style requirements, including those for file naming. The PLOS ONE style templates can be found at

https://journals.plos.org/plosone/s/file?id=wjVg/PLOSOne_formatting_sample_main_body.pdf and

https://journals.plos.org/plosone/s/file?id=ba62/PLOSOne_formatting_sample_title_authors_affiliations.pdf

2.  Thank you for stating the following in the Financial Disclosure section:

"DY was supported by the Korea Medical Device Development Fund grant funded by the Korea government (the Ministry of Science and ICT, the Ministry of Trade, Industry and Energy, the Ministry of Health & Welfare, the Ministry of Food and Drug Safety) (Project Number: 202012B04). (www.https://www.kmdf.org/)

DY was also supported by a Government-wide R&D Fund project for infectious disease research (GFID), Republic of Korea (grant numbers: HG18C0067). (https://www.gfid.or.kr)

The funders had no role in study design, data collection and analysis, decision to publish, or preparation of the manuscript."

We note that one or more of the authors are employed by a commercial company: "BUD.on Inc, Jeonju, Jeollabuk-do, Republic of Korea"

a. Please provide an amended Funding Statement declaring this commercial affiliation, as well as a statement regarding the Role of Funders in your study. If the funding organization did not play a role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript and only provided financial support in the form of authors' salaries and/or research materials, please review your statements relating to the author contributions, and ensure you have specifically and accurately indicated the role(s) that these authors had in your study. You can update author roles in the Author Contributions section of the online submission form.

Please also include the following statement within your amended Funding Statement.

“The funder provided support in the form of salaries for authors [insert relevant initials], but did not have any additional role in the study design, data collection and analysis, decision to publish, or preparation of the manuscript. The specific roles of these authors are articulated in the ‘author contributions’ section.”

If your commercial affiliation did play a role in your study, please state and explain this role within your updated Funding Statement.

b. Please also provide an updated Competing Interests Statement declaring this commercial affiliation along with any other relevant declarations relating to employment, consultancy, patents, products in development, or marketed products, etc.

Within your Competing Interests Statement, please confirm that this commercial affiliation does not alter your adherence to all PLOS ONE policies on sharing data and materials by including the following statement: "This does not alter our adherence to  PLOS ONE policies on sharing data and materials.” (as detailed online in our guide for authors http://journals.plos.org/plosone/s/competing-interests) . If this adherence statement is not accurate and  there are restrictions on sharing of data and/or materials, please state these. Please note that we cannot proceed with consideration of your article until this information has been declared.

Please include both an updated Funding Statement and Competing Interests Statement in your cover letter. We will change the online submission form on your behalf.

3. We note you have included a table to which you do not refer in the text of your manuscript. Please ensure that you refer to Table 3 in your text; if accepted, production will need this reference to link the reader to the Table.

4. Please include captions for your Supporting Information files at the end of your manuscript, and update any in-text citations to match accordingly. Please see our Supporting Information guidelines for more information: http://journals.plos.org/plosone/s/supporting-information.

[Note: HTML markup is below. Please do not edit.]

Reviewers' comments:

Reviewer's Responses to Questions

Comments to the Author

1. Is the manuscript technically sound, and do the data support the conclusions?

The manuscript must describe a technically sound piece of scientific research with data that supports the conclusions. Experiments must have been conducted rigorously, with appropriate controls, replication, and sample sizes. The conclusions must be drawn appropriately based on the data presented.

Reviewer #1: Yes

**********

2. Has the statistical analysis been performed appropriately and rigorously?

Reviewer #1: I Don't Know

**********

3. Have the authors made all data underlying the findings in their manuscript fully available?

The PLOS Data policy requires authors to make all data underlying the findings described in their manuscript fully available without restriction, with rare exception (please refer to the Data Availability Statement in the manuscript PDF file). The data should be provided as part of the manuscript or its supporting information, or deposited to a public repository. For example, in addition to summary statistics, the data points behind means, medians and variance measures should be available. If there are restrictions on publicly sharing data—e.g. participant privacy or use of data from a third party—those must be specified.

Reviewer #1: No

**********

4. Is the manuscript presented in an intelligible fashion and written in standard English?

PLOS ONE does not copyedit accepted manuscripts, so the language in submitted articles must be clear, correct, and unambiguous. Any typographical or grammatical errors should be corrected at revision, so please note any specific errors here.

Reviewer #1: Yes

**********

5. Review Comments to the Author

Please use the space provided to explain your answers to the questions above. You may also include additional comments for the author, including concerns about dual publication, research ethics, or publication ethics. (Please upload your review as an attachment if it exceeds 20,000 characters)

Reviewer #1: I read this paper with interest as it aims to evaluate the high risk issue of drug-induced QT prolongation. The data driven approach described by the authors has merits in that it could overcome limitations of hypothesis-driven investigations that rely on an a priori signal. However, this approach the authors take introduces different biases, notably that infrequently prescribed drugs are not evaluated which could be strongly associated with ADEs.

Suggestions for improvement are described as follows -

1. Others relied on evidence/data from the literature to understand which are more likely to be associated with ADEs including QTc prolongation. What makes your approach unique is you did not limit yourself to those drugs with known associations. I would make this differentiation more clear in the description of candidate drug selection.

2. In the introduction there is discussion that continuous ECG monitoring is better because it allows for better understanding of causality and controls for things such as patient adherence. While this may be true, it limits the generalizability of the findings in real world situations beyond the ICU. In the real-world, outside of ICU settings patient sporadic adherence to meds is important to factor in – whether they doubled the dose or skipped a dose can significantly impact results. Please make it clear RE: what settings your findings are generalizable to and the limitations of this approach.

3. Some results are interspersed within the methods, notably in the study design and population section. Please separate the methods from the results.

4. The methods states drugs of interest are included if they are prescribed more than 500 times and then later it says for >5% of study subjects. Can you please clarify? And please elaborate on how this approach has limitations as well. Although a drug may not be frequently prescribed, its risk of prolonging the QT interval persists and you excluded drugs infrequently prescribed.

5. Please explain why the 18 drugs already classified by credmeds were excluded. Including them may have provided some triangulation to support your findings.

6. Please provide a reference for your definitions of QTc prolongation. Clinical and research audiences often look to >/=500ms as a meaningful measure, but this does not necessarily align with all measures of abnormal QTc intervals.

7. Please explain why the 12 lead data was used in addition to the continuous data.

8. In the methods, please elaborate on the patient population from which 12-lead data was collected. Was is only ICU patients as was the case for continuous data?

9. Please clarify whether the candidate drugs were evaluated before the ECG measurements (not 7 days after).

10. 1st sentence of the 3rd paragraph of the introduction: It is unclear why “Standard 12-lead electrocardiogram (ECG) data” is explicitly called out. The subsequent limitations are not related to the 12-lead (versus continuous).

11. Please be sure you are using the acronym EMR correctly. An EMR and EHR are different and most health systems use an EHR, not EMR.

12. Please consider whether adherence or compliance is the more accurate term to describe how/if patients take medications.

**********

6. PLOS authors have the option to publish the peer review history of their article (what does this mean?). If published, this will include your full peer review and any attached files.

If you choose “no”, your identity will remain anonymous but your review may still be made public.

Do you want your identity to be public for this peer review? For information about this choice, including consent withdrawal, please see our Privacy Policy.

Reviewer #1: No

[NOTE: If reviewer comments were submitted as an attachment file, they will be attached to this email and accessible via the submission site. Please log into your account, locate the manuscript record, and check for the action link "View Attachments". If this link does not appear, there are no attachment files.]

While revising your submission, please upload your figure files to the Preflight Analysis and Conversion Engine (PACE) digital diagnostic tool, https://pacev2.apexcovantage.com/. PACE helps ensure that figures meet PLOS requirements. To use PACE, you must first register as a user. Registration is free. Then, login and navigate to the UPLOAD tab, where you will find detailed instructions on how to use the tool. If you encounter any issues or have any questions when using PACE, please email PLOS at figures@plos.org. Please note that Supporting Information files do not need this step.

3 Jan 2022

We would like to thank Reviewer 1 for the effort to review our paper and for valuable comments which have helped us substantially improve the manuscript. Our responses to the reviewer’s comments are provided below.

Responses to Reviewer 1

Comment #1:

Others relied on evidence/data from the literature to understand which are more likely to be associated with ADEs including QTc prolongation. What makes your approach unique is you did not limit yourself to those drugs with known associations. I would make this differentiation more clear in the description of candidate drug selection.

Response to comment #1:

We would like to thank the reviewer for this suggestion. We agree with this recommendation. Following the comment, we modified the description in the “candidate drug selection” section to clarify our strength as follows:

• (Page 6) “And we included all drugs to evaluate the QT prolongation risk of all drugs, including those not unknown yet, but drugs that were prescribed more than 500 times in the cases were included to secure the statistical power.”

Comment #2 and #7:

#2: In the introduction there is discussion that continuous ECG monitoring is better because it allows for better understanding of causality and controls for things such as patient adherence. While this may be true, it limits the generalizability of the findings in real world situations beyond the ICU. In the real-world, outside of ICU settings patient sporadic adherence to meds is important to factor in – whether they doubled the dose or skipped a dose can significantly impact results. Please make it clear RE: what settings your findings are generalizable to and the limitations of this approach.

#7: Please explain why the 12 lead data was used in addition to the continuous data.

Response to comment #2 and #7:

We appreciate the reviewer's comment. Twelve lead ECG and continuous ECG have their unique advantages and limitations. By using both, we aimed to achieve the advantages of two different data sources and overcome the limitations of each data source. We modified our description to make clear our aim of study design as follows:

• (Page 4) Therefore, in addition to using standard 12-lead ECG data, this study used the continuous ECG data extracted from an intensive care unit (ICU) patient-monitoring device. As the administration of all drugs was recorded in the ICU, we accurately identified the exact time point of drug exposure and the time until the QT prolongation event. Further, using continuous ECG data, we conducted a retrospective cohort study, considered to obtain higher levels of evidence than case-control studies, and investigated acute adverse drug events. However, it has limited generalizability because the continuous ECG was usually measured in an intensive care unit (ICU), and the characteristics of patients and popularly used drugs are different between ICU and general wards.

• (Page 4) To make up for the limitation of standard 12-lead ECG data and the continuous ECG data extracted from an ICU patient-monitoring device, we used both data together in evaluating the QT prolongation risk of drugs.

Comment #3:

Some results are interspersed within the methods, notably in the study design and population section. Please separate the methods from the results.

Response to comment #3:

We appreciate the reviewer's comment. We reorganized the methods and results section. Please see the revised version of our manuscript. We hope that our manuscript is now suitable for publication.

Comment #4:

The methods states drugs of interest are included if they are prescribed more than 500 times and then later it says for >5% of study subjects. Can you please clarify? And please elaborate on how this approach has limitations as well. Although a drug may not be frequently prescribed, its risk of prolonging the QT interval persists and you excluded drugs infrequently prescribed.

Response to comment #4:

Both conditions were used to secure statistical power in each analysis. However, due to the different study designs, different conditions were adopted. In the case of a case-control study, usually, the case group is much smaller than the control group. If there is not enough drug prescription count in the case group, the 95% confidence interval is wide and becomes statistically non-significant even though the odds ratio is high. In retrospective cohort design, if there is not enough portion of exposure group, the comparison between exposure and non-exposure groups cannot be conducted. To clarify these points, we added the following text in the revised version.

• (Page 6) Further, we extracted all prescription data seven days before the date of ECG measurements in the QT prolongation cases, counting the frequency of each drug use. We also included all drugs to evaluate the QT prolongation risk of all drugs, including those not unknown yet. However, drugs prescribed more than 500 times in the cases were included to secure the statistical power.

• (Page 8) To secure at least 5% of the exposure group for further analysis, we selected 78 drugs prescribed for more than 5% of the study subjects as candidate drugs.

We also added the further possibility on the analysis on the drugs excluded in the study in the future as follows:

• (Page 16) Lastly, not all drugs used in the subject hospital were analyzed. It was to secure the least number of patients exposed to the target drug. If the drugs are more used, those drugs will be able to be included in our analysis in the future.

Comment #5:

Please explain why the 18 drugs already classified by credmeds were excluded. Including them may have provided some triangulation to support your findings.

Response to comment #5:

In the retrospective cohort study design, it is critical to reduce potential bias by matching exposure and non-exposure group. To match each group as elaborately as possible, we included the prescription history of those 18 drugs in the matching variable. We added this description in the revised version as follows:

• (Page 8) Eighteen drugs were already classified as QT prolongation risk drugs by CredibleMeds; these are used for propensity score matching (see “Matching exposure and non-exposure groups”). Therefore, we finally analyzed the remaining 60 drugs.

Comment #6:

Please provide a reference for your definitions of QTc prolongation. Clinical and research audiences often look to >/=500ms as a meaningful measure, but this does not necessarily align with all measures of abnormal QTc intervals.

Response to comment #6:

We appreciate the reviewer's comment. As per the comment, QTc>/=500 is also used. But the definition of QTc prolongation according to the American Heart Association (AHA), The American College of Cardiology Foundation (ACCF), Heart Rhythm Society (HRS), QTc longer than 450ms in men and 460 in women is recommended. QTc longer than 500 is usually used to define severe QTc prolongation. We added the following recommendations from AHA/ACCF/HRS and our previous studies used the same QTc definition in the reference list.

• (Page 6) To create the list of candidate drugs for analysis, we extracted all ECG results with QT prolongation (QTc>450 for men and QTc>460 for women) and randomly selected one ECG with several prolonged QT intervals for a single patient [17-19].

• (Page 8) We defined QT prolongation as QTc>450 in men and QTc>460 in women [17-19].

• (References) 17. Rautaharju PM, Surawicz B, Gettes LS, Bailey JJ, Childers R, Deal BJ, et al. AHA/ACCF/HRS recommendations for the standardization and interpretation of the electrocardiogram: part IV: the ST segment, T and U waves, and the QT interval: a scientific statement from the American Heart Association Electrocardiography and Arrhythmias Committee, Council on Clinical Cardiology; the American College of Cardiology Foundation; and the Heart Rhythm Society. Endorsed by the International Society for Computerized Electrocardiology. J Am Coll Cardiol. 2009;53(11):982-91. Epub 2009/03/14. doi: 10.1016/j.jacc.2008.12.014. PubMed PMID: 19281931.

• (References) 18. Choi BJ, Koo Y, Kim TY, Chung WY, Jung YJ, Park JE, et al. Risk of QT prolongation through drug interactions between hydroxychloroquine and concomitant drugs prescribed in real world practice. Sci Rep. 2021;11(1):6918. Epub 2021/03/27. doi: 10.1038/s41598-021-86321-z. PubMed PMID: 33767276; PubMed Central PMCID: PMCPMC7994840.

• (References) 19. Kim TY, Choi BJ, Koo Y, Lee S, Yoon D. Development of a Risk Score for QT Prolongation in the Intensive Care Unit Using Time-Series Electrocardiogram Data and Electronic Medical Records. Healthc Inform Res. 2021;27(3):182-8. Epub 2021/08/14. doi: 10.4258/hir.2021.27.3.182. PubMed PMID: 34384200; PubMed Central PMCID: PMCPMC8369048.

Comment #8:

In the methods, please elaborate on the patient population from which 12-lead data was collected. Was is only ICU patients as was the case for continuous data?

Response to comment #8:

Thanks for this comment. We agree that the description of the study population for 12-lead data was not enough in the previous version. We modified the description to clarify the subject population as follows:

• (Page 5) Since we extracted data regardless of visit or admission type, all patients (outpatients, inpatients admitted to general wards/ICUs, patients visited emergence department, etc.) were included.

Comment #9:

Please clarify whether the candidate drugs were evaluated before the ECG measurements (not 7 days after).

Response to comment #9:

We would like to thank the reviewer for this suggestion. As suggested, we modified the description to clarify the period included in our study as follows:

• (Page 6) Further, we extracted all prescription data seven days before the date of ECG measurements in the QT prolongation cases…

Comment #10:

1st sentence of the 3rd paragraph of the introduction: It is unclear why “Standard 12-lead electrocardiogram (ECG) data” is explicitly called out. The subsequent limitations are not related to the 12-lead (versus continuous).

Response to comment #10:

We agree with this comment. That limitation is not limited to the 12-lead ECG data. Therefore, we modified the first part of the 3rd paragraph of the introduction as follows:

• (Page 3) Many studies have examined drug-induced QT prolongation, but most of these investigations have the following limitations. In many cases, researchers first select the drug to be investigated [5, 8]. This approach enables researchers to investigate the risks for only some drugs of interest.

Comment #11:

Please be sure you are using the acronym EMR correctly. An EMR and EHR are different and most health systems use an EHR, not EMR.

Response to comment #11:

EHR is correct. Thanks for correcting our mistake. We revised them as follows:

• (Page 5) The first comprised standard 12-lead ECG data [11]. We extracted the standard 12-lead ECG data from the ECG MUSE system (GE Healthcare) of Ajou University Hospital for the period between 1994 and 2018 and linked them to electronic health record (EHR) data.

• (Page 7) … we adopted the propensity score matching method to match the control group (subjects whose QTc interval was within normal range) with the case group (subjects whose QTc interval was prolonged) to adjust confounding variables with the following covariates: Gender and age at the ECG examination date, the latest serum potassium and calcium levels calculated within a year of the ECG measurement date, the comorbidities recorded in the EHR within a year of the ECG measurement date…

• (Page 8) For each covariate, all the drug infusions within 12 hours of the index time and all medical diagnosis records made before the index time were extracted from EHR data.

Comment #12:

Please consider whether adherence or compliance is the more accurate term to describe how/if patients take medications.

Response to comment #12:

We would like to thank the reviewer for this suggestion. We noticed that ‘adherence’ has been used as a replacement for ‘compliance’, and ‘adherence’ is a more suitable term because it includes the meaning of patients’ autonomy.

• (Page 4) Since a patient’s drug adherence for the period between two measurements is unknown, it is difficult to capture the exact time gap between drug exposure and adverse drug events and identify acute adverse drug events.

Submitted filename: PO_QT_Response_letter.docx

Click here for additional data file.

13 Jan 2022

Data-driven drug-induced QT prolongation surveillance using adverse reaction signals derived from 12-lead and continuous electrocardiogram data

PONE-D-21-15865R1

Dear Dr. Yoon,

We’re pleased to inform you that your manuscript has been judged scientifically suitable for publication and will be formally accepted for publication once it meets all outstanding technical requirements.

Within one week, you’ll receive an e-mail detailing the required amendments. When these have been addressed, you’ll receive a formal acceptance letter and your manuscript will be scheduled for publication.

An invoice for payment will follow shortly after the formal acceptance. To ensure an efficient process, please log into Editorial Manager at http://www.editorialmanager.com/pone/, click the 'Update My Information' link at the top of the page, and double check that your user information is up-to-date. If you have any billing related questions, please contact our Author Billing department directly at authorbilling@plos.org.

If your institution or institutions have a press office, please notify them about your upcoming paper to help maximize its impact. If they’ll be preparing press materials, please inform our press team as soon as possible -- no later than 48 hours after receiving the formal acceptance. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information, please contact onepress@plos.org.

Kind regards,

Chiara Lazzeri

Academic Editor

PLOS ONE

Additional Editor Comments (optional):

Reviewers' comments:

21 Jan 2022

PONE-D-21-15865R1

Data-driven drug-induced QT prolongation surveillance using adverse reaction signals derived from 12-lead and continuous electrocardiogram data

Dear Dr. Yoon:

I'm pleased to inform you that your manuscript has been deemed suitable for publication in PLOS ONE. Congratulations! Your manuscript is now with our production department.

If your institution or institutions have a press office, please let them know about your upcoming paper now to help maximize its impact. If they'll be preparing press materials, please inform our press team within the next 48 hours. Your manuscript will remain under strict press embargo until 2 pm Eastern Time on the date of publication. For more information please contact onepress@plos.org.

If we can help with anything else, please email us at plosone@plos.org.

Thank you for submitting your work to PLOS ONE and supporting open access.

Kind regards,

PLOS ONE Editorial Office Staff

on behalf of

Dr. Chiara Lazzeri

Academic Editor

PLOS ONE