ConceptWAS: a high-throughput method for early identification of COVID-19 presenting symptoms.

OBJECTIVE: Identifying symptoms highly specific to COVID-19 would improve the clinical and public health response to infectious outbreaks. Here, we describe a high-throughput approach - Concept-Wide Association Study (ConceptWAS) that systematically scans a disease's clinical manifestations from clinical notes. We used this method to identify symptoms specific to COVID-19 early in the course of the pandemic.
METHODS: Using the Vanderbilt University Medical Center (VUMC) EHR, we parsed clinical notes through a natural language processing pipeline to extract clinical concepts. We examined the difference in concepts derived from the notes of COVID-19-positive and COVID-19-negative patients on the PCR testing date. We performed ConceptWAS using the cumulative data every two weeks for early identifying specific COVID-19 symptoms.
RESULTS: We processed 87,753 notes 19,692 patients (1,483 COVID-19-positive) subjected to COVID-19 PCR testing between March 8, 2020, and May 27, 2020. We found 68 clinical concepts significantly associated with COVID-19. We identified symptoms associated with increasing risk of COVID-19, including "absent sense of smell" (odds ratio [OR] = 4.97, 95% confidence interval [CI] = 3.21-7.50), "fever" (OR = 1.43, 95% CI = 1.28-1.59), "with cough fever" (OR = 2.29, 95% CI = 1.75-2.96), and "ageusia" (OR = 5.18, 95% CI = 3.02-8.58). Using ConceptWAS, we were able to detect loss sense of smell or taste three weeks prior to their inclusion as symptoms of the disease by the Centers for Disease Control and Prevention (CDC).
CONCLUSION: ConceptWAS is a high-throughput approach for exploring specific symptoms of a disease like COVID-19, with a promise for enabling EHR-powered early disease manifestations identification.

Introduction

As of October 14, 2020, over 7.7 million people in the United States (U.S.) and 37 million worldwide were infected with coronavirus SARS-CoV-2, the agent responsible for COVID-19 [1]. The virus’s high transmissibility, lack of native immunity, and the dearth of effective treatments make managing COVID-19 uniquely challenging. Hence, early recognition of specific COVID-19 symptoms plays an essential role in the clinical and public health response, enabling rapid symptom screening, diagnostic testing, and contact tracing.

Early in the pandemic, physicians observed fever, cough, and shortness of breath as presenting symptoms of COVID-19; however, these symptoms are common to many viral and bacterial illnesses [2]. Subsequently, as new symptoms were reported, health departments and ministries updated the list of COVID-19 symptoms; for example, the U.S. CDC and the Department of Health and Social Care in the UK added loss of smell or taste, a highly indicative symptom [3] to the list in late April and mid-May, respectively [4,5]. Therefore, methods of identifying the specific COVID-19 symptoms early during a pandemic are required, which is crucial to inform the public on when to present for testing, and potentially reduce the ultimate size of the outbreak, lowering overall morbidity and mortality.

Recent efforts to track COVID-19 symptoms have used methods such as scanning publications or twitter [6,7], deploying questionnaires [8], or releasing apps to self-report symptoms [9]. However, results from publications and questionnaires may be delayed; data from social media or self-reported apps do not always include proper controls and lack physiological assessments to determine the COVID-19 status. Electronic Health Records (EHR) data has also been used to characterize COVID-19, due to the availability of routinely collected medical data. However, existing studies were limited by structured data (e.g., coded diagnoses, procedures, or lab tests) [10,11] and lacked a portable and high-throughput approach [12].

Here, we present a high-throughput approach (ConceptWAS) for early identification of clinical manifestations of COVID-19 using natural language processing (NLP) on EHR clinical notes. By examining the EHR-derived concepts related to patients’ signs or symptoms, ConceptWAS assesses whether any of the concepts are associated with the presence or absence of a disease (e.g., COVID-19). Using this approach, we identified the symptoms that were specific to COVID-19. In particular, we performed ConceptWAS using the cumulative data every two weeks to demonstrate the timeline of emerging symptoms. We also conducted a chart review to validate the significant associations.

Methods

Study setting

The study was performed at Vanderbilt University Medical Center (VUMC), one of the largest primary care and referral health systems serving over one million patients annually from middle Tennessee and the Southeast United States. We used data from patients represented in the VUMC EHR aged ≥18 years. The study was approved by the VUMC Institutional Review Board (IRB #200512).

Cohort definition

We firstly identified patients who received at least one SARS-CoV-2 polymerase chain reaction (PCR) test between March 8 (when the first COVID-19 emerged in VUMC) and May 27, 2020 (Figure A.1). The COVID-19 status was determined using the PCR test result. The case group (COVID-19-positive) was defined as patients who had >=1 PCR positive result, and the control group (COVID-19-negative) consisted of patients with only negative PCR tests. We excluded patients who had no clinical notes on the day the PCR test was ordered.

Data collection

We extracted clinical notes from 24 hours prior to PCR testing date (day0) for COVID-19-positive and negative patients (>86% of patients had at least one note within the time window, see Figure B.1). If a patient first tested negative and then subsequently tested positive or if a patient were tested positive more than once, we used the date of the first positive PCR test as day0. We also segmented the study period into a two-week interval and used the cumulative data every two weeks to perform a temporal analysis.

Concept extraction

We used KnowledgeMap Concept Indexer (KMCI [13]) to preprocess the clinical notes and extract concepts (Figure C.1). KMCI is a local NLP pipeline developed at VUMC for medical notes processing and entity recognition, which has been used for several clinical and genomic studies [13-15]. The preprocessing includes sentence boundary detection, tokenization, part-of-speech tagging, section header identification. The concepts were represented as Unified Medical Language System concept unique identifiers (UMLS CUIs). Since we focused on capturing clinical manifestations of COVID-19, we restricted the concepts to SNOMED Clinical Terms and specific semantic types, e.g., finding, sign or symptom, disease or syndrome, individual behaviors, or mental process (see full list in Table C.1).

Assertion and negation detection

A main challenge of clinical NLP is to accurately detect the clinical entities’ assertion modifier such as negated, uncertain, and hypothetical (e.g. describe a future hypothetical or instruction for patients). We took the following steps to post-process the KMCI output to remove CUIs that were uncertain, and hypothetical. We first excluded any concepts that arose from family history sections. Next, we removed any sentences with future sense or subjunctive mood (e.g. “should”, “could”, or “if”) that describe a hypothetical or instruction for patients. We excluded inquiry sentences that served as the template questions without a simple confirmed answer (e.g. “Yes”, “No”, or “None”) as well. For recognition of negated concepts (e.g. “patient denies having any fever”), KMCI implements NegEx, a widely-used algorithm to detect negations. However, NegEx may miss post-negation triggers such as “Cough: No”. We added regular expression rules based on our local note template to enhance negation detection.

The extended processing modules was implemented using Python 3.6. After processing, the extracted CUIs served as the input for following ConceptWAS analysis.

ConceptWAS analysis

Similar like genome-wide association study (GWAS) and phenome-wide association study (PheWAS) that scan genomic and phenomic data for associations with a given disease or a genetic variant [16,17], conceptWAS examines the clinical concepts retrieved from clinical notes to determine if any concept is associated with a disease. In this study, we applied a conceptWAS to identify associations between symptoms-related concepts and the presence of COVID-19.

We applied Firth’s logistic regression to examine the association for each concept, adjusted by age, gender, and race. We chose Firth’s logistic regression because it has become a standard approach for analyzing binary outcomes with small samples [18]. Negated and non-negated CUIs are treated separately. A CUI with a negation flag (yes/no) is only counted once per patient. Firth’s logistic regression was implemented using R version 3.4.3 and the logistf package. As we tested multiple hypotheses, we used a Bonferroni correction for the significance level. We report the odds ratio (OR), p-values, and the prevalence in case and control groups for each CUI. We used a volcano plot to show p-values and the odds ratio for all CUIs. We also used a forest plot to show the significant concepts that were relevant to signs and symptoms.

Chart review

We performed a manual chart review to evaluate the clinical plausibility of identified signals. We reviewed a CUI if 1) its p-value met Bonferroni-corrected significance, and 2) it was clinically meaningful (e.g., we excluded CUIs such as “finding [CUI C00000243095]” in a sentence like “Findings are nonspecific.”). We randomly selected notes from which the CUI was identified. Two authors (M.E.G. and H.N.E.) with clinical background ascertained whether the identified CUI was a true signal or false positive.

Results

We identified 19,692 patients with COVID-19 PCR test results during the study period (Figure A.1). Patients’ mean age was 45 (44.6 ± 16.9) years. A total of 1,483 (7.5%) patients tested positive for COVID-19. The COVID-19-positive group was younger (41.5 ± 16.2 vs. 44.9 ± 16.9), more often male (48.0% vs. 41.7%), less often white (49.6% vs. 66.7%), and newer to VUMC (EHR length 7.3 years ± 8.1 vs. 9.2 ± 8.5) compared to COVID-19-negative patients (Table 1).

Table 1.

Patient characteristics of the study cohort

Attribute	Cases: COVID-19-positive (n=1,483)	Controls: COVID-19-negative (n=18,209)	P- value
Age (mean years +/− stddev)	41.5 (16.2)	44.9 (16.9)	<0.0001
Gender (% Male)	48.0%	41.7%	<0.0001*
Race (% White)	49.6%	66.7%	<0.0001*
Average EHR length (years, +/− stddev)	7.3 (8.1)	9.2 (8.5)	<0.0001
Average CUIs (+/− stddev)	46.1 (61.1)	71.9 (96.3)	<0.0001

2-proportion z hypothesis test was performed. For age, EHR length, and average CUIs, a t- test was performed for comparing the median and standard deviations.

Comparison of EHR-derived concepts between COVID-19 positive and negative patients

We extracted 87,753 clinical notes for the 19,692 patients. After using the NLP pipeline to processing the notes, we recognized 19,595 CUIs (including negated status) with semantic types of interests (Table B.1). Using ConceptWAS to compare EHR-derived concepts for COVID-19 positive and negative patients, 68 CUIs were identified after adjusting for multiple testing (Bonferroni-corrected significance, P < 2.55E-06) (Figure 1, Table E.1). The top signals included “depression” (OR = 0.34, 95% CI = 0.24–0.47), “edema” (OR = 0.40, 95% CI = 0.29–0.53), “fever (negated)” (OR = 0.63, 95% CI = 0.55–0.72), and “reaction anxiety” (OR = 0.39, 95% CI = 0.28–0.52). Specifically, symptoms concepts associated with COVID-19-positive patients included “absent sense of smell” (OR = 4.97, 95% CI = 3.21–7.50), “fever” (OR = 1.43, 95% CI = 1.28–1.59), “with cough fever” (OR = 2.29, 95% CI = 1.75–2.96), and “ageusia” (OR = 5.18, 95% CI = 3.02–8.58) (Figure 2).

Figure 1.

Volcano plot of a ConceptWAS scan for 19, 692 patients that included COVID-19-positive group (cases) and negative group (controls). The points are colored by the semantic type of the concepts. Selected associations related to signs, symptoms, or diseases/syndromes are labeled. The volcano plot indicates -log 10 (p-value) for association (y-axis) plotted against their respective log 2 (fold change) (x-axis). The dashed line represents significance level using a Bonferroni correction.

Figure 2.

Forest plot comparing individual concepts between COVID-19-positive (case) and COVID-19-negative (control) patients. Selected associations include the significant signals related to semantic types of symptoms that met Bonferroni-corrected significance (p-value < 2.55E-06). The odds ratio has been adjusted for age, gender, and race. The concepts are ordered by p-value.

Concepts related to smoking status such as “current some day smoker”, “former smoker”, and “smoking monitoring status” were more frequently reported in the COVID-negative group than in the COVID-positive group (OR < 1, P < 2.55E-06), suggesting more smokers in control group. To ascertain whether this signal was true or false positives due to wrongly assertion detection by NLP pipeline, we performed a chart review of 80 patients’ notes that had smoking-related CUIs. We found that 79 of 80 patients confirmed an affirmative smoking status (see below chart review).

Temporal analysis

We performed ConceptWAS using the cumulative data every two weeks within the study period (Figure 3, Figure D.1). By week 4 (by April 5, 2020), “absence of smell” (OR = 10.24; 95% CI = 5.18–20.06) and “ageusia” (loss of taste, OR = 11.79; 95% CI = 5.55–25.2) became significantly associated with increased risk of COVID-19 infection. These two signals remained significant through the subsequent weeks (Supplementary Data). Fever (negated) appeared (OR = 0.55; 95% CI = 0.43–0.71) at week 2 (between March 8 and 22, 2020), and “with cough fever” became significant (OR = 2.09; 95% CI = 1.60–2.70) from the 8th week (between March 8 and May 3, 2020). The depression and anxiety appeared significantly starting from week 4 (by April 5, 2020).

Figure 3.

Temporal ConceptWAS using bi-weekly cumulative data. For significant signals (related to signs, symptoms) using all data (labeled in Figure 2), the plot indicates their −log 10 (p-value) for association (y-axis) against using the cumulative data started between March 8, 2020 to n weeks (x-axis). The dashed line indicates a significant association using a Bonferroni correction.

To validate the signals, we reviewed patient’s charts for significant concepts with high clinical relevance. We randomly selected 20 notes for each concept to review whether the notes mentioned the symptoms in the expected attribute (e.g. affirmative or negated) (Table 2). The significant concepts from the ConceptWAS that compared COVID-19 positive and negative patients, such as “absent sense of smell”, “ageusia”, “mental depression”, and concepts related to smoking status (e.g. “current some day smoker”, “former smoker”, and “smoking monitoring status”) were consistent with the expected attribute based on chart review.

Table 2.

Results of chart reviews.

Concepts	Reviewed samples	True signals	True signals percentage %	Examples of false positive
Absent sense of smell	20	19	95.00%	“(−) altered/loss of smell”, were wrongly recognized as an affirmative/ positive attribute.
Ageustia	20	19	95.00%	“Symptoms, n/v, fever, cough, loss of taste or smell or around anyone + for Covid 19.”
Mental Depression	20	18	90.00%	One was recognized from a medical history title without any answers; the other came from a recommendation for further Psychosocial assessment.
Current some day smoker	20	20	100.00%
Smoking monitoring status	20	19	95.00%	One is uncertain. “Smoking Status Not on file”.
Fever	20	17	85.00%	Template issue. “The following ROS were reviewed and are negative, unless otherwise stated as +positive:1 Constitutional: Fever; malaise”
Pericardial Fluid (neg)	20	20	100.00%
Hydrocephalus [neg]	20	20	100.00%
Hydronephroses	20	20	100.00%
Blood group AB Rh(D) negative (finding)	20	0	0.00%	From blood typing tests. This signal was not specific to blood type AB+, but generated by other ABO blood types and Rh-positive patients.
Allergy test positive (finding)	20	5	25.00%	The false positives were wrongly mapped from a sentence like “He /She has been exposed to covid, family member or friends have tested positive.”
Laurin-Sandrow syndrome	20	20	100.00%
Cough nonproductive	20	20	100.00%
In total	260	217	83.46%

Although “smoking monitoring status” was generated by an inquiry term used in a template of a chart, after we post-processed the KMCI output to remove irrelevant concepts and refine negation, the smoking monitoring status followed by a negated answer was recognized as a negated attribute. We reviewed 20 notes that mentioned the “smoking monitoring status (affirmative/positive attribute)” and 19 were either current or former smokers.

We also found false positive concepts, mostly due to NLP entity recognition errors. For example, “additional information” was recognized as “adequate knowledge”. The concept “fever” with positive attribute has three false positives, mainly due to a few specific chart templates used for denoting the negation, which were not captured by NLP pipeline.

Discussion

Our work describes a high-throughput and reproducible approach (ConceptWAS) that use EHR notes to early identify pandemic disease symptoms and investigate clinical manifestations for further hypothesis-driven study. We applied ConceptWAS to a cohort of patients who underwent COVID-19 testing. We replicated several well-known symptoms of COVID-19, such as fever, loss of smell/taste, and cough with fever [19-21]. Using ConceptWAS, we were able to detect the signal of loss of smell and taste as early as April 5, 2020, nearly three weeks earlier than the date that they were listed as COVID-19 symptoms by the CDC [3]. - Our results demonstrate the feasibility of using ConceptWAS for an early detection of symptoms of an unknown disease.

We also observed several signals enriched in the COVID-19-negative group. For example, depression and anxiety have a higher prevalence among patients who tested negative. These signals first became significant starting from April 5, 2020, corresponding to the date when the Governor of Tennessee issued a “safer at home” Executive Order and a “stay at home” order. It reflects the mental health issues that the shutdown and quarantine policies may bring to the people [22,23]. We also find a higher percentage of smoking status concepts in the COVID-19-negative group. Earlier epidemiological studies found that fewer smokers are among COVID-19 patients or hospitalized COVID-19 patients [21,24], which are consistent with our findings of the negative correlation between smoking and COVID-19. One explanation could be the impact of nicotine on ACE-2, as nicotine has been suggested to play a protective role against COVID-19 [25]. It is also possible that smokers are taking greater social precautions because of perceived higher risk for respiratory complications from COVID-19, thus reducing their risk of contracting the virus. Although these findings suggest that smoking may be a protective factor, lack of evidence and known adverse events associated with smoking dissuade continued smoking as a protective measure against COVID-19.

ConceptWAS is open-source, portable, and reproducible. Researchers/Users can choose other NLP pipelines (e.g. MetaMap, CLAMP, cTAKES) [26] for concept extraction and use the derived concepts as the input to ConceptWAS.

Through a proof-of-concept study by applying NLP techniques to identify COVID-19 symptoms, we summarize the lessons learned that may help others apply this method.

A high-throughput, lightweight, and reproducible method is important for pandemic disease. ConceptWAS enables a rapid scan of symptoms using clinical notes. These symptoms provided an initial hypothesis for further investigation and could alert clinicians to pay attention to patients with presenting specific symptoms. Researchers can run the ConceptWAS regularly (e.g., weekly or bi-weekly cumulative data) to track the symptoms changes for a pandemic disease.

Running ConceptWAS needs to be cautious about the distribution of different clinical note types. Different clinical note types are unlike from each other due to their specific clinical usage. They may have variable templates and inconsistent length. Therefore, we recommend that researchers check the distribution of document types between cases and controls to avoid sampling bias.

Although NLP has been used in various medical fields to improve information processing and practice [26-29], recognition of negative and uncertain concepts remains a challenge. We enhanced the detection of uncertain arguments and negated concepts by developing rule-based methods as wrappers for entity-identification generated results. Still, our manual chart review suggest that the outcome is not perfect. For example, some notes mentioned negative concepts like “the following ROS were reviewed and are negative, unless otherwise stated as +positive: Constitutional: Fever; malaise.” Such scenarios are difficult for NLP tools to identify. A combination of machine learning and rule-based approaches may improve the detection.

We also learned and recognized that our study has several limitations. First, the study was performed at a single institution with a limited number of COVID-19 patients. As the pandemic crisis evolves and more patients are tested for SARS-CoV-2 in our healthcare system, our ability to detect COVID-19 and clinical concepts’ associations will continue to improve. Second, this study used data from a limited time (before May 27, 2020). In the future, we will extract notes prior to the test date to study the progression of the symptoms and analyze them along different periods. Lastly, as the performance of an NLP system may vary across institutions and databases [26,30], further studies are necessary to assess the generalizability of our findings.

Conclusion

In this study, we describe a high-throughput approach (ConceptWAS) that systematically scans a disease’s clinical manifestations from clinical notes. By applying ConceptWAS on EHR clinical notes from patients subjecting to a COVID-19 test, we detected loss of smell/taste three weeks prior to their inclusion as symptoms of the disease by CDC. The study demonstrates the potential of the EHR-based methods to enable early recognition of specific COVID-19 symptoms, and improve our clinical and public health response to the pandemic.