Ability of emergency medicine clinicians to predict COVID-19 in their patients.

Severe acute respiratory syndrome coronavirus 2 (SARS-CoV-2 or COVID-19) is frequently encountered in the emergency department (ED). Over a hundred COVID-19 clinical diagnosis and prediction rules have been published in the past two years, but many have incomplete results and are at high/uncertain risk for bias [1,2]. The reported C index estimates ranged between 0.71 and 0.99 for general population prediction models 0.65 to >0.99 in diagnostic models, and 0.54 to 0.99 for prognostic models [1]. Most diagnostic models use a combination of vital signs, comorbidities, age, and imaging [1]. The objective of this study was to assess ED clinician's pretest probability for COVID-19 against the institution's gold-standard SARS-CoV-2 RT-PCR as the reference.

After receiving Mayo Clinic institutional review board approval, ED attending physicians (N = 20) and credentialed advanced practice providers (N = 4) working in the Mayo Clinic ED in Jacksonville, FL, were asked to complete a REDCap survey of their pretest probability (0 (no chance) to 100 (certain)) that their patient had COVID-19 after treating the patient but before the COVID-19 test had resulted. Per institutional guidelines, all patients requiring hospital admission were tested for COVID-19, and all other ED patients had testing done at the provider's discretion. The study took place between June 9, 2020, and November 22, 2020, during which the ED tested 4839 patients, of whom 459 (9.5%) were positive, and 3 were indeterminant. Two hundred sixty-nine surveys (5.6% of tested patients) were completed, with only 245 patients (26 COVID-19 positives, 219 COVID-19 negatives) included in the analysis. Subjects were excluded for the following reasons: seven did not have a COVID-19 test result recorded at the index visit, four had an erroneous date of the ED encounter recorded, three had repeat visits to the ED during the study period (repeat encounters removed), two subject encounters were entered twice, two did not acknowledge informed provider consent, three because the medical record number (MRN) was incorrect or missing, and three had administrative errors. Information on the clinical encounter was collected through retrospective chart review by a study investigator and included demographic and triage data, relevant past medical history, documented COVID-19 test in previous 14 days, laboratory tests, radiology obtained in the ED, critical care consults, and the ED disposition.

Continuous variables were summarized with median and range and compared between COVID-19 negative and positive groups using a Wilcoxon rank sum test. Categorical variables were summarized with number and percentage of patients and were compared between COVID-19 negative and positive groups using Fisher's exact test. For the COVID-19 pretest probability, we also estimated the area under the ROC curve (AUC) along with a 95% confidence interval (CI) regarding ability to predict COVID-19 positivity. We estimated the magnitude of association between COVID-19 pretest probability and COVID-19 positivity by calculating an odds ratio (OR) and 95% CI from an unadjusted logistic regression model. P-values <0.05 were considered statistically significant. Statistical analysis was performed using R Statistical Software.

Baseline patient characteristics are in Table 1 . Compared to COVID-19 negative patients, COVID positive patients were more often non-white (36% vs. 15%, P = 0.010), had a higher temperature (Median: 37.2 vs. 36.8 degree Celsius, P = 0.002), had a higher hemoglobin (Median: 13.5 vs. 12.5, P = 0.045), had a lower WBC (Median: 5.8 vs. 7.8, P = 0.003), and more often had a chest x-ray (92% vs. 58%, P < 0.001). COVID-19 pretest probabilities between COVID negative and positive patients are shown in Table 1. Compared to COVID-19 negative patients, COVID-19 positive patients had a significantly higher COVID-19 pretest probability (Median: 72 vs. 10, P < 0.001, Fig. 1 ), and this corresponded to an AUC of 0.81 (95% CI: 0.72–0.91), indicating relatively good predictive ability. For each 10-unit increase in COVID-19 pretest probability, the odds of a positive COVID-19 test increased multiplicatively by 1.49 (95% CI: 1.30–1.74). Our results are similar to an ED study reporting clinical gestalt for COVID-19 with an AUC of 80.8%, which increased to 91.6% with medical imaging [3]. Another ED study found that clinical gestalt improved their COVID-19 predictive modeling [4].

Table 1

Comparisons of patients positive and negative for COVID-19

	COVID-19 positive patients (N = 26)		COVID-19 negative patients (N = 219)
Variable	N	Median (minimum, maximum) or (%) of patients	N	Median (minimum, maximum) or No. (%) of patients	P-value
COVID-19 pretest probability (continuous)	26	72 (5, 100)	219	10 (0, 98)	<0.001
COVID-19 pretest probability (categorical)
0–20	5	(19.2%)	145	(66.2%)
21–40	1	(3.8%)	28	(12.8%)
41–60	4	(15.4%)	11	(5.0%)
61–80	5	(19.2%)	24	(11.0%)
81–100	11	(42.3%)	11	(5.0%)
Sex (Male)	26	15 (57.7%)	219	121 (55.3%)	0.84
Age (years)	26	60 (18, 86)	219	67 (18, 97)	0.18
Ethnicity (not Hispanic or Latino)	26	25 (96.2%)	211	206 (97.6%)	0.56
Race (non-White)	25	9 (36.0%)	212	31 (14.6%)	0.010
Tobacco use	26	8 (30.8%)	215	106 (49.3%)	0.18
History of Asthma	26	2 (7.7%)	219	17 (7.8%)	1.00
History of CAD	26	5 (19.2%)	219	46 (21.0%)	1.00
History of CHF	26	2 (7.7%)	219	27 (12.3%)	0.75
History of COPD	26	1 (3.8%)	219	25 (11.4%)	0.33
History of DM	26	7 (26.9%)	219	41 (18.7%)	0.31
History of HTN	26	16 (61.5%)	219	121 (55.3%)	0.68
History of Dx Dialysis	26	0 (0.0%)	219	14 (6.4%)	0.37
Temperature (Celsius)	26	37.2 (36.2, 37.8)	217	36.8 (35.7, 39.6)	0.002
Heart rate	26	83 (48, 113)	219	77 (37, 138)	0.19
Systolic blood pressure	26	124 (104, 163)	219	128 (85, 191)	0.42
Diastolic blood pressure	26	75.5 (62, 111)	219	75 (40, 123)	0.86
Respiratory rate	26	18 (12, 27)	216	18 (12, 24)	0.20
Spo2	26	95 (92, 100)	219	96 (90, 100)	0.25
Acuity	26		218		0.13
1		0 (0.0%)		5 (2.3%)
2		3 (11.5%)		69 (31.7%)
3		23 (88.5%)		143 (65.6%)
4		0 (0.0%)		1 (0.5%)
COVID test in previous 14 days	26	0 (0.0%)	219	25 (11.4%)	0.086
Hemoglobin	21	13.5 (3, 16.9)	210	12.5 (3.9, 18.3)	0.045
WBC	21	5.8 (2, 13.7)	210	7.8 (1.3, 86.4)	0.003
Platelets	21	185 (64, 436)	210	221 (12, 965)	0.11
Sodium	21	135 (122, 142)	209	137 (122, 145)	0.074
Potassium	21	4.1 (3.3, 5.1)	209	4.0 (2.2, 7.1)	0.56
Chloride	21	97 (87, 111)	209	100 (86, 112)	0.27
BUN	21	17 (6, 102)	209	17 (4, 122)	0.71
Creatinine	21	0.86 (0.44, 4.67)	209	0.97 (0.23, 15.98)	0.39
eGFR	21	79 (15, 90)	209	65 (0, 90)	0.24
Chest x-ray	26	24 (92.3%)	219	127 (58.0%)	<0.001
CT-Chest/Angio/PE study	26	2 (7.7%)	219	24 (11.0%)	1.00
Critical care consult in ED	26	2 (7.7%)	218	13 (6.0%)	0.67
ED disposition	26		218		0.076
Admit		10 (38.5%)		124 (56.9%)
Admit-OR		0 (0.0%)		1 (0.5%)
AMA		0 (0.0%)		1 (0.5%)
Discharge		14 (53.8%)		59 (27.1%)
Observation		2 (7.7%)		33 (15.1%)

P-values result from a Wilcoxon rank sum test (continuous and ordinal variables) or Fisher's exact test (categorical variables). COVID-19 probability assigned by physicians was able to discriminate between COVID-19 positive and negative patients with an AUC of 0.81 (95% CI: 0.72–0.91).

Fig. 1

Boxplots of COVID pretest probability COVID-19 negative and positive patients.

Clinician gestalt, independent of specific formal scoring tools and specific ancillary testing algorithms, was good for predicating COVID-19 in ED patients. As the ED provider pretest probability for COVID-19 increased, so did the patient's likelihood to test positive. Our results are similar to other more conventional COVID-19 clinical prediction tools and close to ED provider accuracy for pneumonia (80%), group A beta-hemolytic streptococcal pharyngitis (73%), acute coronary syndrome (75%), acute heart failure (86%), pulmonary embolism (81%), and appendicitis (84%) [[5], [6], [7], [8], [9]]. Our study was limited by a small sample size, that data was collected from a single institution from a small percentage of eligible patients—suggesting selection bias, and had a limited number of participating clinicians. We did not discriminate between patients reinfected with COVID-19 or asymptomatic patients testing positive with persistent viral shedding from a recent infection.

Reprints

Johnathan M. Sheele, MD, MHS, MPH, Department of Emergency Medicine, Mayo Clinic, 4500 San Pablo Rd, Jacksonville, FL 32224 (Sheele.Johnathan@Mayo.edu).

Author contributions

J.M.S.: Study concept and design; acquisition, analysis, and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; study supervision.

S·F: Drafting of the manuscript.

A.V.L.: Analysis and interpretation of the data; critical revision of the manuscript for important intellectual content.

M.H.: Analysis and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical expertise.

A.H.: Analysis and interpretation of the data; drafting of the manuscript; critical revision of the manuscript for important intellectual content; statistical expertise.

L.S.: Study concept and design; critical revision of the manuscript for important intellectual content.

Declaration of Competing Interest

All authors report no conflicts of interest.

External funding.

None.