Characterizing Functional Health Status of Surgical Patients in Clinical Notes.

Functional health status is an important factor not only for determining overall health, but also for measuring risks of adverse events. Our hypothesis is that important functional status data is contained in clinical notes. We found that several categories of phrases related to functional status including diagnoses, activity and care assessments, physical exam, functional scores, assistive equipment, symptoms, and surgical history were important factors. Use of functional health status level terms from our chart review compared to National Surgical Quality Improvement Program determination had varying sensitivities for correct functional status category identification, with 96% for independent patients, 60% for partially dependent patients, and 44% for totally dependent patients. Inter-rater agreement assessing term relevance to functional health status was high at 91% (Kappa=0.74). Functional status-related terms in clinical notes show potential for use in future methodologies for automated detection of functional health status for quality improvement registries and other clinical assessments.

Introduction

Functional health status is often defined as one’s ability to perform daily activities required to meet basic needs, fulfill usual roles, and maintain their health and well-being[1]. It is increasingly recognized that a patient’s functional health status is important for determining overall general health and has been used as a factor to estimate preoperative risk of complications and adverse events[2-5].Unfortunately, the measurement and documentation of functional health status is often not standardized particularly for front line clinical practice. Often, physicians and other clinicians use a combination of scoring systems, clinical judgement, and physical exam to determine a patient’s functional health status.

There are several tools created to determine a patient’s functional health status. These include tools like the Karnofsky Performance Scale, an observational method of functional health status determination using a 0-100 point scale[6,7]. Functional health status may also be determined through patient or caregiver-completed assessments such as the Patient-Reported Outcomes Measurement Information System (PROMIS)[8]. Alternatively, functional health status may be determined through calculations as a composite score. One example of this is the Frailty Index, which measures patient medical problems compared to an age appropriate list of medical problems[9]. Metabolic equivalents (METs) are also often used to measure functional capacity, a surrogate of functional health status[10]. There are also frameworks and guidelines created to standardize the determination of a patient’s functional health status, such as the World Health Organization’s International Classification of Functioning, Disability and Health[11].

The American College of Surgeons National Surgical Quality Improvement Program (NSQIP) is a quality improvement registry which collects patient data to track post-operative outcomes and complications. This program offers high quality, risk adjusted data that is nationally validated. Data is collected manually by the surgical clinical reviewer (SCR), a trained nurse with specific training in the registry and its definitions[12]. Participation in this program has been shown to improve surgical outcomes and decrease post-operative morbidity[12,13]. Despite its usefulness, data collection remains labor intensive and participation is expensive[14]. With respect to patient functional health status, the NSQIP trained reviewers most often manually review charts to abstract a determination for the level of the patient’s functional health status placing patients into one of three categories (Table 1): independent, partially dependent, and totally dependent. An independent patient is defined as one who does not require assistance from another person for any activities of daily living, including one who functions independently with the use of prosthetics, equipment, and/or devices. A partially dependent patient requires some assistance from another person for activities of daily living regardless of use of prosthetics, equipment, and/or devices. Finally, a totally dependent patient requires total assistance for all activities of daily living. Functional health status in the NSQIP database is determined within 30 days prior to the operation and is highly correlated with post-operative outcomes[15].

Table 1:

Karnofsky Performance Scale[6,7] and NSQIP functional status scale

Performance Score	Functional Status Performance
100	No complaints, no evidence of disease
90	Able to complete major activities; minor signs and symptoms of disease
80	Normal activity with effort; some signs and symptoms of disease
70	Care of self; unable to carry on normal activities or do active work
60	Requires occasional assistance; able to care for most of personal needs
50	Requires considerable assistance and frequent medical care
40	Disabled; requires special care and assistance
30	Severely disabled; hospital admission is indicated; death not imminent
20	Very sick; hospital admission necessary and active treatment necessary
10	Moribund; fatal processes progressing
0	Death
	NSQIP functional status scale
Independent	Does not require assistance from another person for any activities of daily
	living, including one who functions independently with the use of
	prosthetics, equipment, and/or devices.
Partially Dependent	Requires some assistance from another person for activities of daily living
	regardless of use of prosthetics, equipment, and/or devices.
Totally Dependent	Requires total assistance for all activities of daily living.

By reviewing and categorizing terms and phrases in clinical notes associated with functional health status, it may be possible to improve automated abstraction efforts. Functional health status is often poorly defined, or difficult to define with individual patients between different providers and scoring systems are not uniformly used. As a first step toward understanding the value of the clinical notes for functional health status, we sought to develop a library of terms associated with functional status. More broadly, our hypothesis is that much of the data relating to functional health status would be found within free-text documentation in clinical notes.

Methods

Institutional Review Board approval was obtained for this study. The study was completed at the University of Minnesota Medical Center, an integrated health system in partnership with Fairview Health Services based in Minneapolis, serving the upper Midwest. We performed our initial search using the medical center’s Clinical Data Repository (CDR). Data in from the CDR is compiled from the electronic health records of more than two million patients from eight hospitals and forty clinics[16]. The CDR was queried for all patients included in the NSQIP database over three years (2013-2015). Of these patients, a stratified random convenience sample of 75 patients (twenty-five patients in each of the three functional health status categories as defined by NSQIP: “independent”, “partially dependent”, and “totally dependent”) were selected.

Physician reviewers (SS and EA) were blinded to the NSQIP functional health status determination for each patient. Reviewers examined all clinical notes and forms for the 30 days prior to the operative procedure for which functional health status was originally measured and determined. All phrases associated with functional health status were recorded. Details associated with the phrase, such as clinical note section, type of note, author credentials, and author specialty were also recorded.

After completing the chart review and recording all functional health status data, our reviewers assigned a NSQIP functional health status category and a Karnofsky Performance Score (Table 1) to each of the patient charts based only on functional health status terms recorded during the chart review. These scores were compared with the gold standard scores, which were the determinations previously made in the NSQIP registry by the SCR. To assess the inter-rater reliability of our functional health status determination of associated phrases, a subset of 8 overlapping patients (10.7%) was performed by both reviewers with percentage agreement and Kappa calculated. Statistical analyses were performed using R software (Vienna, Austria, 2017).

Results

A total of 75 patient charts from 2013-2015 were reviewed. In these charts, a total of 1,353 clinical notes were reviewed. Within these clinical notes, there were 1,328 phrases identified which were associated with the determination of a patient’s functional health status. There was a good variety of surgical specialties represented by the operations for which functional status was assessed in the NSQIP registry. Given the interest in functional health status for this study, specialties that operate on problems associated with low functional health status are more highly represented (i.e., neurosurgery, plastic surgery, colorectal surgery, urology, orthopedic surgery). Patient demographic and surgical information is summarized in Table 2.

Table 2:

Patient Demographics and Surgical Specialty

	Median Age (Range)	Male Patients n (%)
All Patients	51.5 (21-91)	39 (52%)
NSQIP Functional Status
Independent	48 (21-77)	11(44%)
Partially Dependent	59.8 (28-91)	13 (52%)
Totally Dependent	47.6 (22-75)	15 (60%)
Surgical Specialty	Charts n (%)
Urology	13 (17.3%)
Orthopedic Surgery	11 (14.7%)
Gynecologic Surgery	10 (13.3%)
Neurosurgery	10 (13.3%)
General Surgery	8 (10.7%)
Colorectal Surgery	7 (9.3%)
Plastic Surgery	5 (6.7%)
Vascular Surgery	5 (6.7%)
Bariatric Surgery	3 (4%)
Otolaryngology	2 (2.7%)
Thoracic Surgery	2 (2.7%)

All of the 1,328 phrases were also annotated according to the type of clinical note in which they appeared as well as the clinical note section in which they appeared. These distinctions were evaluated separately. Breakdown of clinical note type and clinical note section can be found in Table 3. The “progress note” category within the clinical note type includes both daily progress notes written by physicians as well as progress and miscellaneous notes written by nursing and ancillary staff. In some cases, clinical note sections were not clearly delineated or present; these notes are included in the “not applicable” category. This study found that the majority of functional health status information was found in history & physical notes, anesthesia assessments, and office visits, which would be likely to have some component of assessment of patient functional status prior to an operation. The clinical note sections that featured the most functional health status data were history of present illness, assessment/plan, review of systems, and physical exam. All of these sections are likely to describe symptoms, major medical problems, and impairments that affect the patient.

Table 3:

Functional Health Status Phrases by Clinical Note Type, Note Section, and Author

Clinical Note Type	Phrases n (%)
History & Physical	440 (33.1%)
Anesthesia Pre-Operative Assessment	338 (25.5%)
Office Visit	237 (17.8%)
Progress Note	160 (12.0%)
Consultation Note	69 (5.2%)
Emergency Department Visit	51 (3.8%)
Telephone Note	23 (1.7%)
Operative Note	8 (0.6%)
Discharge Summary	2 (0.2%)
Clinical Note Section
History of Present Illness	327 (24.6%)
Not Applicable	215 (16.2%)
Assessment/Plan	199 (15.0%)
Review of Systems	185 (13.9%)
Physical Exam	156 (11.7%)
Past Medical History	141 (10.6%)
Social History	38 (2.9%)
Past Surgical History	26(2.0%)
Chief Complaint	18 (1.4%)
Form Elements	14 (1.1%)
Operative Indications	9 (0.7%)
Type of Author
Staff Physician	795 (59.9%)
Midlevel Provider	182 (13.7%)
Resident or Fellow	144 (10.8%)
Registered Nurse	64 (4.8%)
Other	30 (2.3%)
Physical Therapist	28 (2.1%)
Wound Ostomy Continence Nurse	26 (2.0%)
Occupational Therapist	25 (1.9%)
Medical Assistant	21 (1.6%)
Social Worker	18 (1.4%)
Medical Student	5 (0.4%)

Functional health status-related phrases were recorded in the electronic medical record most frequently by physicians. An analysis of functional health status terms ordered by author role is shown in Table 3. The vast majority of functional health status phrases were recorded by providers (physicians, trainees, midlevel provider).

Author specialty was also recorded. Functional health status data was recorded most frequently by anesthesiologists (23.6% of phrases) and internists (19.7% of phrases). When combined, surgical specialties amounted to 21.7% of the phrases and medical specialties recorded 34.3% of the phrases. Nursing and ancillary staff accounted for 19.7% of functional health status phrases.

Phrases related to functional health status were categorized into seven major categories including: diagnosis, activity/care needs, physical exam elements, functional scores, assistive equipment, symptoms, and surgical history. The phrases are divided according to category and NSQIP functional status determination in Table 4.The amount and proportion of functional health status-related diagnoses increased with increasing level of dependence.

Table 4:

Categorized Phrases According to NSQIP Functional Category

Phrase Category n (%)	Independent n (%)	Partial Dependence n (%)	Total Dependence n (%)	All Patients n (%)
Total Phrases	209	607	512	1328
Diagnosis	23 (11.0%)	194 (32.0%)	255 (49.6%)	472 (35.5%)
Activity/Care Needs	61 (29.2%)	161 (26.5%)	77 (15.0%)	297 (22.3%)
Physical Exam	35 (16.7%)	71 (11.7%)	48 (9.3%)	154 (11.6%)
Elements
Functional Scores	47 (22.5%)	41 (6.8%)	32 (6.2%)	120 (9.0%)
Assistive Equipment	7 (3.3%)	65 (10.7%)	38 (7.4%)	110 (8.3%)
Symptoms	23 (11.0%)	45 (7.4%)	27 (5.3%)	95 (7.2%)
Surgical History	13 (6.2%)	30 (4.9%)	37 (7.2%)	80 (6.0%)

Unique phrases and terms were isolated from all phrases that were recorded during the chart review process. There was a total of 47 unique diagnoses. Unique diagnoses are listed in Table 5. Some of the diagnoses had associated modifiers, which were usually markers of severity or location. For example, the diagnosis “multiple sclerosis” had modifiers “with worsening plaques”, “progressive”, and “unclear”. “Pressure ulcer” had modifiers: “sacral”, “gluteal”, “coccygeal”, and “Stage IV” found in the clinical notes.

Table 5:

Unique Diagnoses

scoliosis	kyphosis	lumbar stenosis	subdural hemorrhage	chronic pain	neurogenic bladder	spinal cord injury	multiple sclerosis
menin-gioma	CNS lymphoma	hemiplegia	monoplegia	autism	learning disability	Mobius Syndrome	paraplegia
ulcer	neurogenic bowel	hip fracture	dementia	malnutrition	Spina Bifida	hydrocephalus	decubitus ulcer
paralysis	Polio	post-Polio syndromes	hyper-reflexia	spasticity	Lyme Disease	Chiari Malformation	weakness
spasmodic dysphonia	limb hypo-genesis	congenital deformity	radiculitis	Cauda Equina	stroke	critical limb ischemia	seizure
Alzheimer Disease	Parkinson Disease	mental retardation	Cerebral Palsy	cognitive defects	dysreflexia	developmental delay

A large portion of patient activity level and care needs were unique; however, the themes of these phrases were similar. Usually, activities measured were similar but there were varying levels of dependence and assistance required between patients. Care needs varied slightly by level of care and nomenclature for facility type. Activity level was often judged based on several activities of daily living: ambulation/walking, eating/cooking, climbing stairs, transferring, toileting, bathing/hygiene, dressing, and exercise. General statements were sometimes made to summarize the patient’s level of activity, such as “not frail”, “good mobility”, or “low exercise capacity”. Care needs were relayed through assistive facility (nursing home, long term care facility, adult foster care, assisted living) or by the individuals helping with daily cares (husband caregiver, personal care assistant, home nurse, daily skilled nursing care).

Physical exam elements were classified into unique terms and phrases. There were 87 unique physical exam elements that could be further divided into three physical exam categories. There were 37 unique “motor/strength/sensation” items, 37 unique “general exam/appearance” items, and 13 unique “cognitive” items.

Functional status scoring systems were particularly useful for making a functional status determination, but were not present for all patients. These scoring systems included American Society of Anesthesiologists (ASA) Class, Karnofsky Functional Scale, Berg Balance Scale, METs estimation, the EQ-5D quality of life questionnaire, and the PROMIS questionnaire. METS and ASA were most prevalent in this chart review and frequently recorded by anesthesiologists, likely reflecting the peri-operative status of these patients.

Unique assistive equipment is recorded in Table 6. There were 31 unique equipment items/devices that were relevant to functional health status.

Table 6:

Unique Assistive Equipment

spinal cord stimulator	indwelling Foley	wheelchair	motorized wheelchair	Baclofen pump	urinary catheters	home oxygen	lift chair
walker	bath bench	leg brace	foot brace	knee brace	torso brace	neck brace	crutches
scooter	intrathecal pump	home ramp	prosthesis	ostomy pouch	Jay cushion	Roho cushion	Hoyer lift
shower chair	cane	BiPAP	hospital bed	stretcher	grab bars	ventilator

Twenty-eight unique symptoms were found in this study. Like diagnoses, modifiers of the symptoms typically highlighted symptom severity, frequency, and location. The unique symptoms related to functional health status determination are found in Table 7.

Table 7:

Unique Symptoms

pain	weakness	shortness of breath	tingling	numbness	spasticity	fatigue	secretion problems
multiple falls	hematuria with cath	urinary incontinence	fecal incontinence	altered sensation	paresis	paresthesias	radiculopathy
worsening gait	swelling	urinary retention	neuropathy	worsening motor function	worsening neurologic status	seizures	unresponsive
combative behavior	constant movement	memory deficit	slurred speech

Finally, there were terms related to surgical procedures that were helpful for functional status determinations. There were 22 unique phrases in the surgical history that helped with determination listed in Table 8.

Table 8:

Unique Surgical Terms

epidural injection	below knee amputation	thoracic spine surgery	artificial urinary sphincter	suprapubic catheter placement	Mitrofanoff	ventriculo-peritoneal shunt	ventriculo-pleural shunt
neck fusion	above knee amputation	tracheostomy	colostomy	craniotomy	urinary diversion	nephrostomy tubes	percutaneous gastrostomy
ileal conduit	Monti	bladder augmentation	urostomy	disarticula-tion	gastro-jejunostomy

Sensitivity was measured for comparing functional status designation using only functional status level phrases identified in the review and NSQIP surgical clinical reviewer determination. Sensitivity decreased as functional status complexity increased. A sensitivity of 96% was obtained for independent patient identification, 60% for identification of partially dependent patients, and 44% for totally dependent patients. Table 9 demonstrates the distribution of the designation of functional status based on the NSQIP SCR designation as the gold standard and functional status level phrases in separate human reviewer designation.

Table 9:

NSQIP Functional Status (Reviewer Designation versus Gold Standard)

p = 3.2201860600803914e-11

The gold standard functional health status designation was also compared against the Karnofsky Performance Scale as determined by independent review (Figure 1). Independent status had a maximum score of 100, median of 90, and minimum of 70 (interquartile range [IQR] 80-100). Partially dependent status had a maximum score of 100, median of 60, and a minimum of 40 (IQR 60-70). Totally dependent patients had a maximum score of 70, median of 50 and minimum of 30 (IQR 40-50).

Figure 1:

Karnofsky Score versus NSQIP Functional Status

Lastly, phrases were assessed for their relevance to functional health status by a second reviewer (EA). Ten percent of the phrases were used for assessment. Inter-rater reliability for determination if a phrase was important for functional health status was scored with an agreement of 90.7% and a Cohen’s kappa of 0.737.

Discussion

At present, functional health status data has not been well integrated into electronic medical records and most clinical workflows in practice. The combination of use of a multitude of functional health status assessments along with using clinical judgement and incomplete/inconsistent documentation make this integration challenging. The purpose of this study was to characterize signals for functional health status in clinical notes to attempt to organize and classify functional health status-related data. This study demonstrated that a variety of phrase categories can be helpful for determining functional health status including diagnoses related to functional health status, activity descriptions, home care needs, physical exam, functional scores, assistive equipment and medical devices, symptoms, and surgical procedures. Anesthesiologists and internal medicine physicians were the most frequent recorders of functional health status data in this study, likely because of the perioperative status of the patient.

There were a large number of unique terms for patient activity level and home care needs. The remaining categories had a relatively small number of unique phrases, but many modifiers related to severity, frequency, and location (particularly for diagnoses and symptoms).

We found that correctly identifying NSQIP functional health status category (gold-standard) with functional health status-related terms was more challenging in our chart review with patients of increasing functional health status complexity. As seen in Table 9, there was still correlation between outcomes, however, there was particularly increased variability in the designation for the “totally dependent” category. This could be because functional health status is inherently difficult to classify. Alternatively, it is often difficult to make a determination based on descriptions of a complex topic such as functional health status which can span many different observation categories, particularly when different providers’ descriptions may not align. There were two cases that were classified as “totally dependent” by the SCR, yet “independent” by human reviewer. Because of the wide discrepancy, these cases were re-reviewed and found to have conflicting data in clinical notes. In these patients, there were signals of total/partial dependency for functional health status in some notes, while other providers’ notes clearly stated that the patients were completely independent. This highlights the complexity of functional health status and limitations associated with designating functional health status retrospectively.

This study is limited in its retrospective nature. Also, there are likely additional factors that were involved in the determination of functional health status level by the physicians performing the assessment that were not entered in the electronic medical record. As such, determining functional status in a prospective manner in a standard and rigorous fashion is most ideal. Additionally, this is a single institution study with a study of relatively small sample size, and the description of functional health status may differ between institutions or even between surgical services. In our cohort, several author types (physical therapist, occupational therapists, and social workers) may have been particularly underrepresented. These author types wrote 1.2% of the total clinical notes reviewed, yet contributed to 5.3% of the functional health status-related phrases. It is likely that these authors have a larger impact on the functional health status descriptors than represented in this study. This study reviewed surgical patients, which may have contributed to the lack of documentation from these authors. Perhaps the reason for low percentages of these notes overall is that for cases where patients were totally independent or totally dependent (n=50), physical therapy, occupational therapy, and social work may have been minimally involved, since the supportive needs were either very low, or conversely, maximal and established. Finally, certain surgical services more involved in the care of lower functional status patients were likely over-represented.

Other methods have been used to attempt to characterize functional health status. Another retrospective review was performed which collected functional health status terms within the Veteran Affairs electronic medical record and patient-reported functional health status data on social media, concluding that standard terminologies such as Unified Medical Language System do not sufficiently cover functional health status information[17]. In a separate project, the same group used topic modeling as a method to extract relevant frailty information in clinical text[18].A study by Ruggieri et al. found that functional health status terminologies centered around verb phrases, particularly descriptions of motion. A “frame-semantic” method was used for functional health status representation with a final goal of improving information abstraction and natural language applications of functional health status[19].

Similar work characterizing clinical note data has been performed at our institution to develop an automated method of data abstraction for determination of surgical site infections and other complications[20,21]. Using keywords related to these complications improved the accuracy of the detection of these complications. We identified and categorized terms and phrases used in surgical site infection descriptions in a previous study[22]. Translating this method to this current project, the new terms identified in this study could be valuable for the automated detection of functional health status level. Additionally, these terms could potentially be incorporated into an automated approach to examine notes as they are being written to determine if they meet minimum documentation requirements for functional health status. We believe that expansion of our previous work in an analogous fashion can improve automated detection techniques for functional health status determinations.

Conclusion

Functional health status is a difficult clinical entity to quantify. Determination of level of functional status likely differs between providers, and while functional status scores are often helpful in this determination, they are not always documented. Factors in functional health status determination can be found in clinical notes through diagnoses, activity and home care descriptions, physical exam elements, functional scores, assistive equipment, symptoms, and surgical procedures. The phrases identified in this study could potentially be used to assist in automation of detection of functional health status level.