Rapid Exclusion of COVID Infection With the Artificial Intelligence Electrocardiogram.

OBJECTIVE: To rapidly exclude severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2) infection using artificial intelligence applied to the electrocardiogram (ECG).
METHODS: A global, volunteer consortium from 4 continents identified patients with ECGs obtained around the time of polymerase chain reaction-confirmed COVID-19 diagnosis and age- and sex-matched controls from the same sites. Clinical characteristics, polymerase chain reaction results, and raw electrocardiographic data were collected. A convolutional neural network was trained using 26,153 ECGs (33.2% COVID positive), validated with 3826 ECGs (33.3% positive), and tested on 7870 ECGs not included in other sets (32.7% positive). Performance under different prevalence values was tested by adding control ECGs from a single high-volume site.
RESULTS: The area under the curve for detection of acute COVID-19 infection in the test group was 0.767 (95% CI, 0.756 to 0.778; sensitivity, 98%; specificity, 10%; positive predictive value, 37%; negative predictive value, 91%). To more accurately reflect a real-world population, 50,905 normal controls were added to adjust the COVID prevalence to approximately 5% (2657/58,555), resulting in an area under the curve of 0.780 (95% CI, 0.771 to 0.790) with a specificity of 12.1% and a negative predictive value of 99.2%.
CONCLUSION: Infection with SARS-CoV-2 results in electrocardiographic changes that permit the artificial intelligence-enhanced ECG to be used as a rapid screening test with a high negative predictive value (99.2%). This may permit the development of electrocardiography-based tools to rapidly screen individuals for pandemic control.

The world is currently suffering from a global pandemic caused by the transmission of the severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2), resulting in coronavirus infectious disease 19 (COVID-19). The SARS-CoV-2 virus enters cells when its spike protein binds to angiotensin-converting enzyme 2 (ACE2) receptors, which are richly expressed in the heart.1, 2, 3 Animal data from rabbits as well as human clinical reports indicate that the coronavirus frequently may enter myocytes and, by direct or indirect mechanisms, causes myocardial inflammation, which may in turn be reflected by nonspecific electrocardiographic changes.4, 5, 6, 7 Whereas the myocardiac picture may often be subclinical, elevated troponin levels and electrocardiographic and heart rhythm changes have been frequently observed.8, 9, 10, 11

We have previously demonstrated that artificial intelligence (AI) in the form of convolutional neural networks applied to the electrocardiogram (ECG) can detect subtle, subclinical patterns in an ECG to identify the presence of occult and impending cardiovascular diseases, including left ventricular dysfunction, intermittent atrial fibrillation, and other conditions and demographic characteristics, such as age and sex.12, 13, 14, 15 In this context, we hypothesized that COVID infection would lead to recognizable changes in the AI-enhanced ECG (AI-ECG) and that absence of those changes could exclude the presence of acute coronavirus infection, facilitating point-of-care screening. Given the periodic shortages of reagents for current coronavirus genetic screening tests, delays in obtaining results, and associated costs, a low-cost, readily scalable solution for rapid point-of-care screening is critical for pandemic management.

To test the hypothesis that an electrocardiography-based test could exclude COVID-19, a global volunteer consortium was formed to gather electrocardiographic and clinical data from individuals with and without COVID-19 to build a neural network to detect infection and to make the network widely available.

Methods

Site Coordination

A total of 28 sites from 14 countries on 4 continents were included in this study (Figure 1 ; Appendix 1, available online at http://www.mayoclinicproceedings.org). Each site ensured compliance of participation in this study, including deidentification of ECGs according to local Institutional Review Board policies and specific national and institutional patient privacy guidelines. All sites were given case report forms to complete on all cases, with data collated in a central database (Research Electronic Data Capture [REDCap]; Vanderbilt University).

Figure 1

Geographic distribution of enrolling sites. Shown is the geographic distribution of sites contributing electrocardiograms.

Electrocardiogram Acquisition and Aggregation

All ECGs were aggregated in their raw digital form from local ECG servers. On identification of cases and controls, unique identifiers were applied to each patient at the site, and the raw digital files were transferred to an independent, password-protected research server at Mayo Clinic in Rochester, Minnesota, in FDA-XML format.

Control Population

Three control populations were used. The first population was composed of ECGs from COVID-positive patients that were acquired more than 2 days before COVID-positive polymerase chain reaction (PCR) analysis. The second consisted of patients with ECGs acquired before September 2019. This date was chosen to ensure that the likelihood of any patient’s having been infected with COVID-19 was negligible, given that the first infections recorded occurred in the timeframe of November to December 2019 in Wuhan, China. The first 2 were used for training. The third consisted of 50,905 ECGs from a single site obtained before 2019. These were used to enrich the testing data set by altering the prevalence of COVID-19 to mirror a more general screening setting where the test positivity rate would be expected to be in the 5% to 10% range. All patients (outpatients and inpatients) were included in all sets.

Case Population

The case population consisted of patients with a positive PCR test result for COVID-19. All ECGs available both before and after diagnosis were included in the data transferred from all sites. The date of each ECG and PCR test was recorded in the case report forms, as were the age and sex of the patient. The ECGs acquired in the window of interest served as positive samples, with the window starting 2 days before the positive PCR test result and ending 14 days after it. The ECGs from COVID-positive patients recorded before the window of interest were used as controls (labeled negative samples), and ECGs collected after the window of interest were used for a secondary analysis. Additional data recorded included the World Health Organization (WHO) symptom severity at the time of initial COVID-19 symptoms, at the date of index ECG (closest to first positive PCR test result), at 30 days after index ECG (which also recorded mortality status), and at the point of most severe symptoms during the interval between the index ECG and 30 days after the index ECG (Figure 2 A). The WHO symptom severity scale is summarized in Table 1 .

Figure 2

A, Definition of control, index case electrocardiograms (ECGs), and postinfectious ECGs included in analysis. B, Receiver operating characteristic curves for detection of acute COVID-19 infection from a 12-lead, 6-lead, and 1-lead ECG. AUC, area under the curve; PCR, polymerase chain reaction.

Table 1

World Health Organization Score

1	Not hospitalized, no limitations on activities
2	Not hospitalized, limitation on activities
3	Hospitalized, not requiring supplemental oxygen
4	Hospitalized, requiring supplemental oxygen
5	Hospitalized, on noninvasive ventilation or high-flow oxygen devices
6	Hospitalized, on invasive mechanical ventilation
7	Death

Data Quality and Model Development

Data Quality

To exclude ECGs with extreme noise and artifacts, ECGs with a maximum amplitude of 5 mV or more were excluded. The ECGs with less than 10 seconds worth of data were excluded, as were those with incorrect format or arrival after the analysis completion date.

Model Development

Standard convolutional neural network , and residual neural network architectures were evaluated with multiple initial learning rates (1e-3, 3e-4) and batch sizes (16, 32, and 64). The learning rate was configured to decrease by a factor of 1/3 if the model accuracy did not improve over 3 consecutive epochs. The model with the best area under the curve (AUC) in the internal validation set was selected as the optimal model and tested on the holdout cohort, composed of patients not used in model training or validation. The model architecture was similar to one previously reported (Supplement 1, available online at http://www.mayoclinicproceedings.org).

To evaluate the potential for use of the algorithm with smartphone-based form factors, we also trained models with a single lead (lead I) and with 6 leads (all limb leads: I, II, III, aVF, aVR, aVL) from the 12-lead ECG using a similar network architecture. When the 12-lead and 6-lead inputs are used, 4 of the leads are augmented leads that do not contain additional information. However, to conform to standard electrocardiography devices, we used all 12 leads during the development of the networks. In addition, we created architectures with 8 leads (that use only the independent data found in a 12-lead ECG) and 2 leads (to match the independent data found in 6-lead systems) and found similar results.

Statistical Analyses

The ECGs were partitioned into 3 mutually exclusive sets at the patient level. Model training was conducted on the training set and hyperparameter optimization on the validation set. All performance measures presented are based on the test data, which were not used for training or optimization. The network reports a score between 0 and 1 (0, low likelihood of infection; 1, high likelihood) for each ECG that is evaluated. The AUC was the primary outcome measure and was determined for the test cohort and then with additional controls to assess the impact of prevalence on test performance. The latter represents the scenario in which widespread testing at a lower disease prevalence is conducted, such as screening patients ahead of a medical evaluation or screening asymptomatic passengers as part of the boarding process.

Standard measures of diagnostic performance and their associated confidence intervals were computed according to the Standards for Reporting Diagnostic Accuracy criteria. , To form decisions based on model output, an optimal threshold was selected to provide 99% sensitivity in the validation data. This threshold was used for both the test data alone (∼33% COVID-19 prevalence) and the test data enriched with additional control data (∼5% COVID-19 prevalence). With this threshold, sensitivity, specificity, positive predictive value (PPV), and negative predictive value (NPV) were determined.

To better understand model performance, exploratory analyses across a few key descriptive variables were undertaken. First, COVID-19 prevalence was strongly associated with heart rate. To address this, model performance stratified by heart rate groupings was evaluated. Second, it was hypothesized that increased viral load leads to downstream complications and would be associated with a more pronounced electrocardiographic signature. To explore this concept, we analyzed model prediction scores according to WHO symptom severity score and the change of scores over time in COVID-19 patients with multiple ECGs during follow-up. Finally, to test for an association of model performance and the days between the ECG acquisition and the COVID-19 diagnosis, a linear mixed model was used to examine the slope of the model output over the days surrounding diagnosis. This model included a main (fixed) effect for days from diagnosis and a random subject effect. Confidence intervals, when presented, for measures of diagnostic performance assume that multiple ECGs within a patient were statistically independent. Statistical analyses and model development were conducted using Python version 3.7.6 (Python Software Foundation) and R version 3.5.2 (R Foundation for Statistical Computing).

Role of the Funding Source

The study was designed and conceived by Mayo Clinic investigators with no financial support from industry and made possible through the generous contribution of data, time, human resources, and intellectual capital from medical centers around the world (authors and Discover Consortium) invited to participate (Supplement 2, available online at http://www.mayoclinicproceedings.org). In addition, General Electric, SHL Telemedicine, Philips, and Epiphany Healthcare donated resources, expertise, and, in some cases, equipment to aggregate electrocardiographic data into a central research server for analysis. The study was managed by an international volunteer steering committee (Appendix 2, available online at http://www.mayoclinicproceedings.org). The protocol was approved by Institutional Review Boards at each participating site, with Mayo Clinic in Rochester, Minnesota, serving as the coordinating site.

Results

Characteristics of the Patients and Geographic Distribution

A total of 48,186 valid ECGs of 11,770 patients were included from 28 participating sites in 12 countries across 4 continents (Figure 1). There were 15,117 ECGs obtained from 4419 controls (mean 3.4±7.1 per patient). An additional 32,971 ECGs were obtained from 7340 COVID-19–positive patients (4.5±5.8), with 13,247 obtained close to the time of positive PCR test result (2 days before and up to 14 days after). Characteristics of patients from REDCaps are in Table 2 .

Table 2

Characteristics of Patientsa,b,c

	Control (n=1420)d	COVID-19 (n=982)d	Total (N=2402)d	P value
White				.04
No	248 (52.5)	333 (58.8)	581 (56.0)
Yes	224 (47.5)	233 (41.2)	457 (44.0)
Black or African American				.91
No	212 (41.4)	244 (41.1)	456 (41.2)
Yes	300 (58.6)	350 (58.9)	650 (58.8)
Asian				.15
No	363 (94.3)	450 (96.4)	813 (95.4)
Yes	22 (5.7)	17 (3.6)	39 (4.6)
Native Hawaiian or Pacific Islander				.37
No	377 (100.0)	462 (99.8)	839 (99.9)
Yes	0 (0.0)	1 (0.2)	1 (0.1)
Other race				.94
No	317 (81.5)	384 (81.7)	701 (81.6)
Yes	72 (18.5)	86 (18.3)	158 (18.4)
Health care worker				<.001
No	991 (98.7)	449 (93.3)	1440 (97.0)
Yes	13 (1.3)	32 (6.7)	45 (3.0)
Acute hypoxic respiratory failure (non-ARDS)				<.001
No	131 (56.7)	152 (41.0)	283 (47.0)
Yes	100 (43.3)	219 (59.0)	319 (53.0)
Acute liver injury				.24
No	195 (94.7)	274 (91.9)	469 (93.1)
Yes	11 (5.3)	24 (8.1)	35 (6.9)
Acute myocardial infarction				.41
No	194 (95.1)	282 (96.6)	476 (96.0)
Yes	10 (4.9)	10 (3.4)	20 (4.0)
Acute renal failure requiring hemofiltration				.92
No	195 (94.2)	281 (94.0)	476 (94.1)
Yes	12 (5.8)	18 (6.0)	30 (5.9)
Acute renal injury, no hemofiltration				.57
N-Miss	1195	657	1852
No	169 (75.1)	237 (72.9)	406 (73.8)
Yes	56 (24.9)	88 (27.1)	144 (26.2)
ARDS				.08
No	187 (90.8)	260 (85.5)	447 (87.6)
Yes	19 (9.2)	44 (14.5)	63 (12.4)
Bacteremia				.76
No	199 (97.5)	288 (98.0)	487 (97.8)
Yes	5 (2.5)	6 (2.0)	11 (2.2)
Bacterial pneumonia				.20
No	190 (92.7)	268 (89.3)	458 (90.7)
Yes	15 (7.3)	32 (10.7)	47 (9.3)
Cardiac arrest				.66
No	198 (96.6)	286 (97.3)	484 (97.0)
Yes	7 (3.4)	8 (2.7)	15 (3.0)
Cardiac arrhythmia: atrial fibrillation				.45
No	192 (91.9)	278 (93.6)	470 (92.9)
Yes	17 (8.1)	19 (6.4)	36 (7.1)
Cardiac arrhythmia: heart block				.78
No	201 (99.5)	288 (99.3)	489 (99.4)
Yes	1 (0.5)	2 (0.7)	3 (0.6)
Cardiac arrhythmia: torsades de pointes				1.00
No	202 (100.0)	291 (100.0)	493 (100.0)
Cardiac arrhythmia: ventricular tachycardia				.55
No	196 (97.5)	286 (98.3)	482 (98.0)
Yes	5 (2.5)	5 (1.7)	10 (2.0)
Myocarditis				.71
No	200 (99.0)	289 (99.3)	489 (99.2)
Yes	2 (1.0)	2 (0.7)	4 (0.8)
Pneumothorax				.79
No	202 (99.5)	291 (99.3)	493 (99.4)
Yes	1 (0.5)	2 (0.7)	3 (0.6)
Pleural effusion				.91
No	173 (96.1)	236 (96.3)	409 (96.2)
Yes	7 (3.9)	9 (3.7)	16 (3.8)
Rhabdomyolysis or myositis				.07
No	201 (99.5)	285 (97.3)	486 (98.2)
Yes	1 (0.5)	8 (2.7)	9 (1.8)
Seizure				.65
No	199 (98.5)	288 (99.0)	487 (98.8)
Yes	3 (1.5)	3 (1.0)	6 (1.2)
Sepsis				.70
No	187 (84.6)	266 (83.4)	453 (83.9)
Yes	34 (15.4)	53 (16.6)	87 (16.1)
Shock				.32
No	191 (91.4)	267 (88.7)	458 (89.8)
Yes	18 (8.6)	34 (11.3)	52 (10.2)
Stroke				.64
No	200 (98.5)	286 (97.9)	486 (98.2)
Yes	3 (1.5)	6 (2.1)	9 (1.8)

ARDS, acute respiratory distress syndrome.

Only for subset of patients with REDCap data for each category.

Values are reported as number (%). Boldface P values represent statistical significance.

Reported numbers are lower than those used in the model development as only those patients in whom case report form data were completed are included here.

Because of data format issues, 4647 ECGs from 1 site were excluded . An additional 1028 ECGs had a maximum absolute amplitude of more than 5 mV and were excluded because of potential data quality issues (examples in Supplement 3, available online at http://www.mayoclinicproceedings.org). There were 791 ECGs from 196 COVID-positive patients excluded because of a missing PCR date, and 4710 ECGs from COVID-positive patients collected more than 14 days after the index PCR were excluded from the main analysis as prespecified. The final analyzed cohort consisted of 37,131 ECGs from 10,762 patients, with 13,247 ECGs labeled COVID positive. The distribution of WHO symptom severity scores at the time of index ECG from the 4392 patients in whom the score was recorded is shown in Supplemental Figure 1 (available online at http://www.mayoclinicproceedings.org).

Network Performance: 12 Leads, 6 Leads, and 1 Lead

Based on the study population prevalence of 34.9%, the AUC was 0.767 (95% CI, 0.756 to 0.778) for the 12-lead AI-ECG in identifying acute COVID-19 infection. The model had a sensitivity of 98.0%, specificity of 10.2%, PPV of 36.7%, and NPV of 90.5% and F 1 score of 53.4%. The AUC for the 6-lead algorithm was 0.754 (95% CI, 0.742 to 0.765), with sensitivity of 97.9%, specificity of 7.8%, PPV of 36.1%, and NPV of 87.2% and F 1 score of 52.7%. Finally, the single-lead model resulted in an AUC of 0.745 (95% CI, 0.733 to 0.756) with a sensitivity of 98.5%, specificity of 6.5%, PPV of 35.9%, and NPV of 89.1% (Figure 2B).

To better understand performance, the model was reevaluated by enriching the control population in the testing set with additional patients to vary the disease prevalence in the testing set. The AUC in the enriched cohort using prevalence of 5% was 0.780 (95% CI, 0.771 to 0.790), with 98.0% sensitivity and 12.1% specificity; the NPV is estimated to be 99.2%. With a COVID prevalence of 5% (the Centers for Disease Control and Prevention designation for red, or the point at which more aggressive social restrictions may be needed), for 1000 patients screened, 116 would be reassured of not being infected. Table 3 presents additional estimates of the test performance over a range of disease prevalences.

Table 3

Estimated Model Performance Measured by Positive and Negative Predicative Values Over a Range of COVID Prevalence Values

Disease prevalence	True cases (per 1000)	Negative cases (per 1000)	Positive predictive value (%)	Negative predictive value (%)	Expected positive test results	Expected negative test results
1.0	10	990	1.1	99.8	880	120
2.0	20	980	2.2	99.7	881	119
5.0	50	950	5.5	99.1	884	116
10.0	100	900	11.0	98.2	889	111
15.0	150	850	16.4	97.2	894	106
20.0	200	800	21.8	96.0	899	101
25.0	250	750	27.1	94.8	904	96
30.0	300	700	32.3	93.4	909	91
35.0	350	650	37.5	91.8	914	86

Serial Electrocardiographic Analysis in Patients With COVID-19

For COVID-19 patients who had ECGs available before the index diagnosis, the average network output of their pre–COVID-19 ECGs was similar to that of the control cohort, whereas the distribution of scores after diagnosis was markedly shifted toward larger values, as would be expected on the basis of the AUC for the model (Figure 3 A). Among 321 patients with serial ECGs between –2 and 14 days of diagnosis (2657 ECGs), there was gradual rise in the model output (beta=0.0033/d; P=.002; Supplemental Figure 2, available online at http://www.mayoclinicproceedings.org). During a longer time, a general decrease in the network output was observed up to more than 2 months after their index diagnosis (Figure 3B). There was nonsignificant trend toward higher network scores in patients with higher index WHO symptom severity scores (Figure 4 ). Consistent with this finding, there was a trend toward higher network scores in inpatients vs outpatients (Supplemental Figure 3, available online at http://www.mayoclinicproceedings.org).

Figure 3

A, Average network score among the general control population (control), prediagnosis electrocardiograms (ECGs) from patients with COVID-19 who had ECGs available from before their index COVID-19 diagnosis, and ECGs around the time of COVID-19 diagnosis. Presented are violin plots that indicate the relative proportion of patients composing the final mean and median for each group. B, Serial network scores over time (see text for details). Shown is the average network score for individual groups of patients who had multiple ECGs during follow-up (before index COVID-19 diagnosis and up to more than 2 months after). Presented are violin plots with superimposed box plots. The violin plots provide a visual of the density (relative frequency of values) over the range of the values observed, with the larger width indicating an increased frequency. The box plot shows the lower quartile, median, and upper quartile of the distribution.

Figure 4

Shown is the change in mean network score among electrocardiograms available for patients according to their World Health Organization (WHO) severity score. Higher severity scores were associated with a statistically significant higher detection score.

Impact of Specific Electrocardiographic Features on Network Prediction

Given the anecdotal observation that elevated heart rate at the time of diagnosis might predict infection, we sought to evaluate whether specific heart rate ranges among the broader ECG cohort had an impact on overall model accuracy. Specifically, the network was validated against ECGs within specific heart rate ranges (eg, 70 to 80, 80 to 90), and no significant relationship to model performance was observed (Supplemental Figure 4, available online at http://www.mayoclinicproceedings.org).

Discussion

Main Findings

We found that patients infected with SARS-CoV-2 develop electrocardiographic changes identified by the AI-ECG. If validated prospectively, these may permit the AI-enhanced ECG to be used as a screening test to exclude acute infection. Specifically, assuming a population with 5% coronavirus infection, the NPV of the electrocardiographic screen was 99.2%, which might enable 12% of individuals to proceed without any additional screening. With additional prospective network training and coupling to data from a single PCR test, the number of passed tests may further increase. Through identifying those at low risk of active infection, it can further help in identifying those patients in whom a genetic or antigen-based COVID-19 test may be useful, independent of symptoms, and thus serve as an integral part of a cascaded testing regimen. This proof-of-concept, retrospective analysis demonstrates biologic plausibility, in support of prospective studies.

Importantly, we selected a point on the receiver operating characteristic curve with a high sensitivity and low specificity (Table 4 ) to permit infection exclusion rather than inclusion. This has important practical implications in that the test output is best considered negative (no infection) or indeterminate (further testing needed). Thus, AI-ECG COVID screening applied in this manner must be part of tiered testing that includes PCR point-of-care testing. Whereas this first iteration of the AI-ECG COVID screen is a global test, applied in a similar manner to all individuals, in future iterations, the AI-ECG output could be coupled to the result of a home PCR test to identify a given individual’s COVID-negative AI-ECG signature. Such an approach, if validated, would further enhance the AI-ECG COVID screen performance.

Table 4

Diagnostic Performance at Candidate Thresholds

Threshold	Sensitivity (%)	Specificity (%)
0.44	70.15	70.44
0.05	99.51	4.65
0.10	97.63	11.90
0.15	94.99	20.57
0.20	91.83	30.40
0.25	88.52	40.36
0.30	84.04	49.27
0.35	79.34	58.00
0.40	74.37	65.47
0.45	68.80	71.62
0.50	62.74	77.51
0.55	56.00	82.90
0.60	48.70	87.42
0.65	40.12	91.13
0.70	31.80	94.33
0.75	23.64	96.39
0.80	15.66	98.06
0.85	8.73	99.24
0.90	3.12	99.74
0.95	0.45	100.00

This work used 12-lead ECGs, making these findings particularly useful in the clinic or hospital. However, we found that the AI-ECG worked with use of only 1 lead or 6 leads. Given that the AI-ECG may be acquired with smartphone-enabled electrodes to permit data acquisition in nearly any environment without disrobing, that no body fluids or reagents are needed for the test, and that it can be performed in less than 30 seconds, with proper validation such a test may allow health care systems, businesses, and societies at large to efficiently and effectively mitigate exposure risk through a readily scalable, noninvasive, real-time, low-cost test.

Such a noninvasive method to detect acute but potentially subclinical infection is of particular importance, given the prolonged incubation period (10 to 14 days) and the large proportion of patients who remain asymptomatic but potentially infectious. , While PCR testing continues to evolve, allowing at-home or saliva tests, most still require an unpleasant (and potentially difficult to self-administer) nasal swab or a prescription from a clinician and thus face challenges for broad, societal screening. Furthermore, the turnaround time on PCR testing for COVID-19 ranges from 15 minutes to more than 48 hours, depending on the assay and testing facility. An immediate, accurate, point-of-care “rule out” test would allow a portion of the population to continue to engage in society (whether attending classes at school, visiting restaurants, or going into work). The finding that the AI-ECG performs well with only 6 leads or a single lead suggests that currently available smartphone-enabled form factors may be used for screening. Such devices may be sanitized quickly and do not require removal of clothing or adhesive patches and are inexpensive enough to permit individual ownership. This approach could substantially improve on current entryway screening techniques, such as questionnaires and temperature assessment, which have a limited performance profile.21, 22, 23

Impact of COVID-19 on the ECG

Coronaviruses may have a direct impact on both cardiac function and electrophysiology. , Investigators have demonstrated that rabbit coronavirus infection may result in several electrocardiographic changes, including ST-segment abnormalities, rhythm disturbances, and conduction defects that appear to be secondary to the myocardial disease induced. Specific to COVID-19, it has also previously been demonstrated that activation of ACE2 may have a direct impact on repolarization vis-à-vis the QT interval, with ACE2 activation also shortening the cardiac action potential in rat and other animal models. , COVID-19 infection has effects on the QT interval, independent of potential QT-prolonging agents. , Moreover, COVID-19 infection results in a plethora of ubiquitous systemic and cellular changes, including severe inflammation and RAS activation, known to affect cardiac repolarization. Thus, it stands to reason that acute COVID-19 infection may have a direct impact on the ECG, in subtle, multifactorial ways. Acute electrocardiographic changes may result from a combination of compensatory changes associated with infections in general (eg, sinus tachycardia), secondary effects on cardiac structure and hemodynamics due to respiratory compromise (eg, right ventricular enlargement or decrease in function), or direct interaction of COVID-19 with the ACE2 receptor (eg, evidence of myocardial injury, inflammation, or changes in ventricular repolarization). Thus, such electrocardiographic changes may help in risk stratifying for potential active COVID-19 infection.

Impact of Disease Prevalence

An important consideration in this study is the impact of prevalence on the NPV and PPV (Table 3). At a prevalence of 33%, which may be reflective of the most severe spikes during the course of the pandemic, the NPV of the algorithm was 95%. At a lower prevalence of 10%, this increased to more than 99%. In making an effort to use this algorithm clinically, it will be important to consider the population prevalence in the context of result interpretation.

Limitations

It is possible that the presence of fever or acute respiratory disturbances, irrespective of causative organism or mechanism, may be driving the network beyond any direct effects attributable to the SARS-CoV-2 virus. Further research comparing ECGs from patients with other types of infectious disease (eg, influenza) and COVID-19 may help elaborate this. However, a rapid screen for active infection may prove useful irrespective of cause, and the model performed similarly well across a range of heart rates. Most patients have an ECG recorded in association with an emergency department visit or hospitalization. Although many patients were not hospitalized (Figure 4), we do not know how many ECGs were obtained in an emergency department. Whether this test can be used to screen outpatients with minimal or no symptoms requires a prospective study. Given the heterogeneity of the population of patients, it is possible that the use of drugs that have an impact on the ECG (eg, hydroxychloroquine) may also have affected network output. However, the diverse global population receiving a wide range of therapies and treatments enhances network robustness and mitigates potential biases. Information about drug use at the time of ECG acquisition was not available for analysis. Clinical characteristics for many patients were not available, particularly for the controls—a consequence of performing an unfunded study carried out by medical volunteers in the midst of a pandemic and of international privacy regulations. The fact that some controls were obtained from patients 2 days before a COVID-positive PCR test result raises the possibility that some controls may have been infected. This may weaken test performance. Other control ECGs were obtained before September 2019, a different time period, potentially introducing confounding or other bias.

A general limitation with neural networks is the lack of explainability, in that the specific electrocardiographic features affecting output are not known, with the theoretical concern that methods of data acquisition or testing may be susceptible to systematic error. We were unable to identify any single dominant feature that robustly characterized network performance. Nonetheless, the reproducible performance across populations from diverse geographies suggests that the tool is robust and could be appropriately used. In addition, whereas PCR is the current standard of care for identifying SARS-CoV-2 infection, the sensitivity is estimated at only 70%, and thus it is unclear how the ECG would perform among infected patients whose PCR test result is negative. Finally, spectrum bias is a possibility in our cohort. There were few asymptomatic patients in the training and validation sets, so further validation of the algorithm on a nonhospitalized or asymptomatic population is necessary.

A practical challenge for the analysis was developing a statistical plan to estimate the confidence intervals for measures of diagnostic performance, particularly receiver operating characteristic AUC, with a wide range of cluster sizes. In particular, across all 48,186 ECGs available for analysis, cluster sizes (ie, multiple ECGs) ranged from 1 to 133, with a median of 2 and a mean of 4. Thus, whereas there was potential for overestimation of the precision with the data (ie, confidence intervals being too narrow), this likelihood would be low considering that the intraclass correlation coefficient of the model predictions was 0.57, and the change in the standard error of a proportion changes little beyond a sample size of 2000. Conservatively, a single representative ECG could be selected for each person and the precision of estimated confidence intervals would be less than ±1.3 percentage point (calculation assumes an effective sample size of 6000 and a proportion of 0.50). This relative range of precision was observed broadly over many of the diagnostic performance measures tabulated for this study. Given that this work represents the primary development of the algorithm and validation studies would be required to use the algorithm in practice, the primary tables and text focus on point estimates.

Conclusion

Infection with SARS-CoV-2 results in electrocardiographic changes that may permit the AI-enhanced ECG to be used as a screening test with a high NPV (99.2%). This may permit the development of electrocardiography-based tools to rapidly screen individuals for pandemic control, especially in a clinic or hospital setting. Development of mobile technology–enabled AI-ECGs may have broader implications that may enable resumption of normal operations across society.