Free-Text Documentation of Dementia Symptoms in Home Healthcare: A Natural Language Processing Study.

BACKGROUND: Little is known about symptom documentation related to Alzheimer's disease and related dementias (ADRD) by home healthcare (HHC) clinicians.
OBJECTIVE: This study: (1) developed a natural language processing (NLP) algorithm that identifies common neuropsychiatric symptoms of ADRD in HHC free-text clinical notes; (2) described symptom clusters and hospitalization or emergency department (ED) visit rates for patients with and without these symptoms.
METHOD: We examined a corpus of -2.6 million free-text notes for 112,237 HHC episodes among 89,459 patients admitted to a non-profit HHC agency for post-acute care with any diagnosis. We used NLP software (NimbleMiner) to construct indicators of six neuropsychiatric symptoms. Structured HHC assessment data were used to identify known ADRD diagnoses and construct measures of hospitalization/ED use during HHC.
RESULTS: Neuropsychiatric symptoms were documented for 40% of episodes. Common clusters included impaired memory, anxiety and/or depressed mood. One in three episodes without an ADRD diagnosis had documented symptoms. Hospitalization/ED rates increased with one or more symptoms present.
CONCLUSION: HHC providers should examine episodes with neuropsychiatric symptoms but no ADRD diagnoses to determine whether ADRD diagnosis was missed or to recommend ADRD evaluation. NLP-generated symptom indicators can help to identify high-risk patients for targeted interventions.

Introduction

Alzheimer’s disease and related dementias (ADRD) represent a looming public health crisis, affecting roughly 5 million people and 11% of older adults in the United States (US) (Alzheimer’s Association, 2013). Older adults with ADRD tend to be clinically complex; roughly half have three or more chronic conditions (Lin et al., 2013). They are at elevated risk for avoidable hospitalizations for chronic conditions, hospital readmission, urinary tract infection, sepsis, and other adverse outcomes (Daiello et al., 2014; Lin et al., 2013; LinNeumann, 2017; Shen et al., 2012), which may be preventable with timely care and effective care (Amjad et al., 2018; Bellantonio et al., 2008; Naylor et al., 2014; Samus et al., 2014, 2015, 2017, 2018).

Patients with ADRD benefit from home-based care that can help to manage symptoms, reduce safety issues and decrease caregiver stress (D’Souza et al., 2015; Gitlin et al., 2018; Lau et al., 2019; Samus et al., 2014, 2017). In the home healthcare (HHC) setting, nurses are uniquely situated to detect symptoms and provide tailored interventions for ADRD patients. However, HHC nurses may be unaware that a patient had an established diagnosis of ADRD. This might happen due to inadequate transfer of information on the patient’s cognitive status prior to entering HHC and the fact that ADRD may not be directly related to the primary reason for the HHC episode (Sockolow et al., 2018, Sockolow, Bass, et al., 2019; Sockolow, Le, et al., 2019). In addition, little is known about HHC clinician practices related to documentation of their patients’ cognitive status. Information in the electronic health record (EHR) can provide insights into HHC nurses’ knowledge of patients’ prior ADRD diagnoses, observations of cognitive symptoms and caregiver needs, and related interventions.

Natural language processing (NLP) is a body of methods for systematic analysis of free-text content. It can be used to explore untapped, yet critical EHR data found in clinical notes. NLP is being increasingly used in healthcare, with up to 80% of information stored as free text (e.g., progress/follow-up notes, admission/discharge summaries, radiology reports) (Ford et al., 2016; Hassanpour & Langlotz, 2016; Meystre et al., 2008). Since ADRD involves a range of behavioral and emotional signs and symptoms, complex family dynamics, and a long trajectory of changes over time (Brookmeyer et al., 2002; Fisher et al., 2016; Gitlin et al., 2017, 2018; Sloane et al., 2017; Sperling et al., 2011; Stansfeld et al., 2018; Werner et al., 2017), it is potentially rich in narrative that HHC nurses may document with varied vocabularies and both explicit and implicit references in the notes. Moreover, observations of patients in their home environment may highlight ADRD signs, symptoms, and unmet needs that may be difficult to observe in other healthcare settings, such as the hospital or an outpatient medical office, outside of the context of daily routines. Although NLP could be useful in identifying clinicians’ observations of these signs and symptoms, it has been underutilized in ADRD research within the HHC setting.

This exploratory study aimed to demonstrate the applicability of NLP in identifying ADRD symptoms as documented by HHC clinicians in a large body of free-text clinical notes extracted from the EHR of a large, non-profit HHC organization. We developed a NLP vocabulary to examine the prevalence of six common neuropsychiatric symptoms of ADRD: depressed mood or apathy; agitation; aggression; anxiety; impaired memory; and delusions or hallucinations (Assal & Cummings, 2002; Siafarikas et al., 2018; Vik-Mo et al., 2018). In addition, we described symptom clusters and compared hospitalization or emergency department (ED) visit rates between patients with and without documented neuropsychiatric symptoms. This work lays a foundation for a comprehensive NLP study of ADRD-related documentation patterns, which could ultimately be used to develop systematic ways to identify HHC patient needs for cognitive evaluation, facilitate a timely diagnosis, help patients and caregivers access services to address symptom management and safety needs, and introduce disease-modifying interventions for those who may benefit (Barnett et al., 2014; Brooker et al., 2014; Dubois et al., 2016; Kirson et al., 2016; Knopman et al., 2000; Mattos et al., 2018; Woods et al., 2019).

Methods

Study data

This study used a large corpus of roughly 2.6 million HHC notes (n = 1,149,586 visit notes and n = 1,461,171 care coordination notes) documented in the EHR of the largest non-profit HHC organization in the United States, located in [blinded for peer review]. Notes were documented by HHC clinicians (e.g., nurses, physical/occupational therapists, social workers, etc.) using the agency’s EHR during or after a patient visit. Visit notes ranged from lengthy admission notes (often written by a registered nurse) to shorter progress notes (e.g., nurse follow-up notes, physical therapy progress notes, etc.). Care coordination notes included documentation of communication with interdisciplinary care team members (e.g., primary care physician, social work, etc.), orders of supply or equipment (e.g., oxygen, wheelchair, etc.), and other care-related information. The average visit note length was 150 words, while the average length of the care coordination notes was 99 words.

Patient hospitalization and ED use were extracted from the Outcomes Assessment and Information Set (OASIS). The OASIS assessment is federally mandated of all home health agencies certified to accept the Centers for Medicare and Medicaid Services (CMS) payments. OASIS assessments are conducted at the patient’s HHC admission and discharge; we used the discharge assessment to capture the study outcomes. Specifically, we used OASIS item “M2300: Emergent Care” that captures patient hospitalization or emergency department visits during the HHC episode.

The OASIS was also used to identify episodes with a known ADRD diagnosis indicated as an ICD-9 code in the payment diagnosis fields upon HHC admission (in the primary or any secondary diagnosis position). The ICD-9 codes used to identify ADRD were derived from the Centers for Medicare and Medicaid Services (CMS) Chronic Condition Warehouse definition of ADRD [Alzheimer’s Disease ICD-9: 331.0; Related Dementias ICD-9: 331.1, 331.11, 331.19, 331.2, 331.7, 331.82; 290.0, 290.1, 290.10, 290.11, 290.12, 290.13, 290.20, 290.21, 290.3, 290.40, 290.41, 290.42, 290.43, 291.2, 294.0, 294.1, 294.10, 294.11, 797] (Chronic Condition Data Warehouse, 2018).

Study sample

The corpus of narrative notes was associated with a retrospective cohort of 89,459 patients with any diagnoses admitted for post-acute HHC services at the study organization during 2014 (1/1/2014–12/31/2014) and living in New York City. Since an individual patient can be admitted and discharged multiple times, the sample of 89,459 patients was associated with a total of 112,237 HHC episodes from which narrative notes were extracted. The patient sample had a mean age of 70.8, was 60.8% female, and 37% lived alone. The sample was racially and ethnically diverse, with 43% white, 27% Black or African-American, 24% Hispanic/Latino, and 6% Asian.

NLP methods

Identifying common neuropsychiatric symptoms

First, we conducted a comprehensive search in research literature databases (e.g., the Cumulative Index to Nursing and Allied Health Literature [CINAHL], PubMed, Google Scholar, etc.) to identify studies focused on neuropsychiatric symptoms among people with ADRD (Assal & Cummings, 2002; Siafarikas et al., 2018; Vik-Mo et al., 2018). This helped us to identify the most frequently reported symptoms, including: (1) depressed mood or apathy; (2) agitation; (3) aggression; (4) anxiety; (5) impaired memory; and (6) delusions or hallucinations.

Next, we identified a preliminary list of terms and expressions for each of the six symptom categories using a large standardized health terminology called the Unified Medical Language System (UMLS; Sinha et al., 2013). UMLS aggregates many other terminologies (e.g., the International Statistical Classification of Diseases and Related Health Problems [ICD] (WHO, 2014), the Systematized Nomenclature of Medicine - Clinical Terms [SNOMED-CT] (SNOMED, 2016), the International Classification for Nursing Practice [ICNP®], (Nurses, 2018) etc.) and compiles lists of synonyms from multiple terminologies. For example, UMLS’s concept “Aggressive behavior” [ID C0001807] has 18 unique synonyms, such as “Aggression”, “Aggressive”, “Aggressive/violent behavior”, etc. In total, 304 UMLS synonyms for the six categories were extracted and added to neuropsychiatric symptoms synonym list.

NLP system development and testing

NimbleMiner was used to develop the NLP algorithms. NimbleMiner is an NLP system previously applied in several clinical domains (Blumenthal et al., 2019; Topaz, Murga, Bar-Bachar, et al., 2019), including HHC (Topaz, Murga, Gaddis, et al., 2019). NimbleMiner includes number of methodological stages of clinical text processing that are briefly described in the Supplementary material (Appendix A).

Study datasets for vocabulary exploration

NimbleMiner enables its users to identify vocabularies of terms and expressions describing a specific concept of interest using a module we call “Rapid vocabulary explorer.” The module relies on language models – numerical representations of multiple text documents (Mikolov et al., 2013) – to help users find the relevant vocabularies. We used two large collections of text documents for rapid vocabulary explorer module. First, we used a computer algorithm (word embedding; Mikolov et al., 2013) to learn a language model (word2vec, more details in Appendix A) from all −2.6 million homecare notes available for this study. Second, to potentially expand the vocabulary beyond language available in clinical notes, we downloaded a large collection of article abstracts from PubMed (a search engine of peer-reviewed biomedical and life sciences literature). To obtain the abstracts, we searched PubMed using query terms for each of the six neuropsychiatric symptoms, which resulted in 215,945 article abstracts (limited to English language abstracts). These abstracts were downloaded and processed by NimbleMiner to create an additional language model. The two models were queried independently by each of the three users using the rapid vocabulary explorer module. Experts included two masters-level registered nurses (RNs) with more than 10 years’ experience in HHC and one PhD-level RN with expertise in informatics. The interrater agreement between the reviewers in terms of retrieved terms and expressions was relatively high (Kappa statistics = .75).

NLP system testing

We used a high likelihood sampling approach to create a testing set (also called gold-standard) of clinical notes as follows. First, we identified a subset of patients admitted to HHC with ADRD diagnoses based on the structured data (indication of ADRD diagnoses on the patent co-morbidity list, n = 10,661, 9.5% of the total sample). Among these patients, we extracted a random subset of 400 clinical notes (50% care coordination notes and 50% visit notes). Each note was annotated by two of the expert reviewers for the presence of one or more of the six neuropsychiatric symptoms. The interrater agreement was relatively high (Kappa statistics = .84), indicating strong agreement between reviewers (McHugh, 2012). All disagreements were reviewed and adjudicated by the PhD-prepared RN.

Next, we applied our NLP system on the gold standard testing set and for each category calculated precision (defined as the number of true positives out of the total number of predicted positives), recall (the number of true positives out of actual number of positives), and F-score (the weighted harmonic mean of the precision and recall).

Symptom cluster evaluation

Once tested, our NLP system was applied on the full sample of clinical notes available for the study. Each note was labeled by the NLP system for the presence of one or more of the six neuropsychiatric symptoms. Labels were aggregated to the HHC episode level, for example, whether the specific HHC episode had mentions of one or more of the six neuropsychiatric symptoms. We labeled cases at the episode level because individual patients might have had several episodes but not necessarily present with symptoms at each episode. Symptom prevalence and clusters (i.e., appearance of several symptoms during the same episode) were summarized and visualized in an UpSet plot to present intersecting sets (Lex et al.,2014). In addition, we compared the distribution of symptoms between episodes with ADRD diagnoses indicated on the OASIS assessment upon HHC admission vs. episodes without ADRD diagnoses.

Comparing hospitalization and ED visit rates

We used structured data from OASIS to summarize hospitalization or ED use during HHC episode. Specifically, we compared the prevalence of having either a hospital admission or an ED visit among patients with no neuropsychiatric symptoms documented in the narrative notes versus patients with one or more symptoms.

The study was approved by the Institutional Review Board of the Visiting Nurse Service of New York.

Results

NLP system performance

The vocabulary exploration phase of NLP enabled the discovery of 1,333 additional terms and expressions compared to 304 terms and expressions extracted from the UMLS, indicating more than four-fold vocabulary expansion (Figure 1).

Figure 1.

Vocabulary exploration.

Most of the vocabulary expansion can be attributed to unique terms and expressions used in clinical notes versus the standard vocabularies included in the UMLS. For example, the category “Impaired memory” had UMLS terms such as “forgetful,” “poor memory,” “mild memory loss,” etc. In addition, the NLP system heled us to discover more nuanced and clinically used terms such as “requires frequent reminders,” “very poor historian,” “can’t recall,” etc. Additional terms and expressions also included misspellings and abbreviations, such as “frogetful,” “fogetful,” “stml [short term memory loss],” etc.

NimbleMiner showed good overall NLP symptom identification performance (overall F-score = .88) when tested on the gold standard set of 400 HHC notes. Table 1 provides performance metrics for each symptom category.

Table 1.

NLP System Performance in Identifying Symptoms.

Domain	Precision	Recall	F-score
Depressed mood or apathy	0.86	0.93	0.89
Agitation	0.95	0.88	0.91
Aggression	0.90	0.96	0.93
Anxiety	0.76	0.93	0.82
Impaired memory	0.84	0.90	0.86
Delusions or hallucinations	0.89	0.85	0.87
Average performance	0.87	0.91	0.88

Symptom distribution and association with hospitalization/ED use

Table 2 presents the overall prevalence of neuropsychiatric symptoms of dementia. In total, 40% of episodes had at least one symptom, with the most frequent being impaired memory (21%), anxiety (16%) and depressed mood or apathy (10%). Table 2 also presents exemplary quotes from HHC notes describing neuropsychiatric symptoms. The selected examples illustrate the diversity in the vocabulary clinicians used to describe each symptom.

Table 2.

Prevalence of Neuropsychiatric Symptoms (N = 112,237 Total Episodes of Care) and Exemplary Quotes from Clinical Notes.

Symptom	N of clinical notes with symptom	N of episodes with symptom	% of episodes with symptom	Example quote 1	Example quote 2
Depressed Mood or Apathy	21,795	11,066	10	Patient reports feeling depressed, unable to fall asleep.	States that he feels tired and disinterested in the things he used to enjoy.
Agitation	7,289	4,028	4	Patient refused to allow RN to check her BP; patient bent her arm and was agitated.	pt[patient] extremely irritable today unable to make progress in teaching, retry next visit.
Aggression	13,242	8,276	7	Patient aggressive and refusing services.	Patient verbally abusive and uncooperative.
Anxiety	33,751	17,558	16	Patient appears weak, anxious and impulsive.	Noted breathing short with exertion due to anxiety.
Impaired Memory	49,619	23,342	21	Patient is a little forgetful, requires reinforcement with teaching reminders.	Patient is a poor historian w stml [short term memory loss].
Delusions or Hallucinations	2,388	982	1	pt reports hearing voices.	Presents with baseline mental status paranoid and delusional.
Any symptom	114,411	45,210	40	—	—

Figure 2 illustrates the prevalence of neuropsychiatric symptoms and symptom clusters by HHC episode. The most prevalent symptom clusters included impaired memory with other symptoms, including: impaired memory with anxiety (n = 2,724 HHC episodes); impaired memory with depressed mood or apathy (n = 1,778 HHC episodes); and impaired memory with anxiety and depressed mood or apathy (n = 1,318 HHC episodes). Other prevalent symptom clusters were anxiety + depressed mood or apathy (n = 1,692 HHC episodes) and anxiety + aggression (n = 686 HHC episodes).

Figure 2.

Prevalence of neuropsychiatric symptoms and symptom clusters by HHC episode.

Figure 3 shows that in episodes with ADRD diagnoses, neuropsychiatric symptoms were documented twice more frequently (73% ADRD episodes had at least one symptom) compared to episodes without ADRD diagnoses (37% ADRD episodes had at least one symptom). Finally, Figure 4 depicts the prevalence of HHC episodes with one or more neuropsychiatric symptoms documented in the narrative notes and describes the rates of hospitalization or ED visit trends. Overall, about 12% of HHC episodes had more than one symptom. Rates of having either a hospitalization or ED visit increased significantly when one or more symptoms were present; a chi-square test showed a statistically significant difference between the groups (p < .05). For example, 14% of patients with no symptoms had a hospitalization or ED visit, while almost half of the patients with five or more symptoms had a hospitalization or ED visit during their HHC episodes.

Figure 3.

Comparison of neuropsychiatric symptom documentation by presence of ADRD diagnoses.

Figure 4.

Rates of having either a hospital admission or ED visit by number of neuropsychiatric symptoms.

Discussion

This study’s findings demonstrate the utility of NLP in identifying selected neuropsychiatric symptoms of ADRD as documented by HHC clinicians in the narrative notes for a population of HHC patients admitted for post-acute care with any diagnosis. The NLP methodology allowed us to identify symptoms documented in a range of categories and expressed with varied terms and phrases. This approach proved beneficial in generating a four-fold expansion of the initial vocabulary which was identified through review of the literature and standardized medical terminology systems (such as the UMLS). NLP methods also enabled us to identify observations that would otherwise be missed in a basic search due to misspellings and abbreviations, allowing for more comprehensive capture of dementia-related information. This exploratory work establishes a foundation for developing a comprehensive NLP algorithm to detect ADRD symptoms, related interventions, and clinician documentation of established diagnoses in the narrative notes and integrating this algorithm into the HHC electronic health record for clinical applications.

The findings also provide preliminary evidence that NLP can be used to identify combinations of dementia symptoms and the most common combinations in a general population of HHC patients. The neuropsychiatric symptoms examined here are not on their own unique to ADRD, nor do they equate with diagnostic criteria. However, further investigation of the symptom clusters and their associations with other clinical, functional, and behavioral characteristics could provide insight into patient needs along the ADRD trajectory and possible warning signals indicated in the narrative notes.

Moreover, the associations found between symptom count and hospitalization or ED use suggests that NLP could be used to identify potential predictors of adverse outcomes for which ADRD patients are at heightened risk (Daiello et al., 2014; Lin et al., 2013, 2017; Shen et al., 2012). This finding warrants further research using predictive modeling techniques to examine whether NLP-derived symptom indicators could be used to identify ADRD patients with particularly high risk for adverse outcomes and design targeted interventions to prevent or mitigate these risks. These efforts could potentially support the goal of improving quality of life for those with ADRD and reducing caregiver stress.

Given that ADRD entails a wide range of cognitive, social, emotional and behavioral symptoms, it is an area ripe for NLP development that could be used in HHC clinical practice in several ways. For patients with an established ADRD diagnosis that is known to the HHC team, the ability to flag symptom observations may help to guide the plan of care to address patient and caregiver needs and potentially reduce the risk for preventable hospitalization and ED use.

Additionally, some patients may have an established ADRD diagnosis that is not known to the HHC team, due to inadequate information transfer across healthcare settings. In this study, we found that more than one in three HHC episodes without an ADRD diagnosis exhibits neuropsychiatric symptoms. For these patients, HHC organizations could develop a workflow in which the HHC team would communicate with the patient’s referring physician and/or primary care provider to determine whether they are missing previous ADRD diagnostic information. This type of workflow would target groups of patients with specific combinations of symptoms that may be associated with ADRD, identified through further analysis of the symptom clusters shown in Figure 2. Finally, for patients who exhibit early symptoms of ADRD but have not been diagnosed previously, flagging symptoms documented in narrative notes may help to identify needs for cognitive evaluation and facilitate timely diagnosis of ADRD.

Study limitations

This study has several important limitations. First, we focused on applying NLP on a subset of common neuropsychiatric symptoms; further work is needed to expand the ADRD symptom categories. In addition, our study used data from one large HHC agency; further work is needed to validate our findings with data from other HHC agencies. Finally, the study data was limited to EHR data within the HHC setting. A more complete capture of ADRD diagnoses within the study cohort might be possible with linkages to data sources for healthcare encounters outside the HHC setting, such as claims data from office-based or inpatient services.

Conclusion

This study was the first to apply NLP to identify common neuropsychiatric symptoms documented by HHC clinicians. We found that neuropsychiatric symptoms were documented for 40% of HHC episodes and common symptom clusters included impaired memory, anxiety and/or depressed mood or apathy. Our results also showed that more than one in three HHC episodes without an ADRD diagnosis had documented neuropsychiatric symptoms. Although these symptoms are not unique to ADRD, they may signal a need for comprehensive neuropsychiatric evaluation. This would help to determine whether the individual is showing early signs of ADRD or cognitive symptoms related to other conditions, thus facilitating timely diagnosis and potential interventions to support the individual and their caregiver(s). In the absence of more complete medical history information, identifying these symptoms in the narrative notes may also prompt the HHC team to communicate with the patient’s other healthcare providers to determine whether the individual has an existing ADRD diagnosis that may have been missed in the HHC plan of care. Incorporating this information into the plan of care could improve the ability of the HHC team to effectively provide routine HHC interventions, such as medication management and falls prevention. Finally, symptoms identified via NLP can be used to prioritize patients at risk for hospitalizations/ED visits for timely interventions.

This study demonstrated the utility of NLP in identifying a core set of neuropsychiatric symptoms that are common among individuals with ADRD. The initial NLP vocabulary developed in this study could be further expanded and refined to include additional signs, symptoms, and indications of unmet needs among HHC patients with ADRD or who may be on a path toward ADRD diagnosis. Upon further development, a NLP algorithm could be integrated into the HHC electronic health record to identify ADRD-related needs that might otherwise go undetected. Thus, NLP may be a powerful tool in improving care coordination, early detection, risk management, and in turn, quality of life for patients and caregivers along the ADRD trajectory.

Click here for additional data file.

Supplemental material, NLP_appendix_A_ADRD for Free-Text Documentation of Dementia Symptoms in Home Healthcare: A Natural Language Processing Study by Maxim Topaz, Victoria Adams, Paula Wilson, Kyungmi Woo and Miriam Ryvicker in Gerontology and Geriatric Medicine