Indicators of missing Electronic Medical Record (EMR) discharge summaries: A retrospective study on Canadian data.

INTRODUCTION AND
OBJECTIVES: Electronic medical records (EMRs), specifically the discharge summary (DS), can improve secondary use data availability and interprofessional communication. We aimed to describe the completeness of our EMRs by assessing the presence of a DS in the EMR. Additionally, we assessed for indicators of a missing DS.
METHODS: A chart review was conducted on 3,011 non-obstetric adult inpatient charts in Calgary, Alberta. 893 charts were missing an electronic DS. A 10% sample was drawn to evaluate the presence of a paper DS. A Chi-square test, Fisher's test and logistic regression measured the associations between electronic DS absence and i) patient and hospital characteristics, and ii) patient comorbidities.
RESULTS: The univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of </1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes, and renal disease were associated with a missing DS. Those that were middle aged, surgical patients, or had fewer comorbidities were more likely to have a missing DS. Within the 10% sample, approximately 50% of all patients were from a surgical department, all of which were missing both electronic and paper discharge summaries.
CONCLUSIONS: Our study describes indicators of missing electronic DS. The DS impacts interprofessional communication, patient outcomes, and data quality. Therefore, the implications of an incomplete DS are widespread. Our findings will caution future researchers using EMR data about the potential for incomplete data, particularly for patients who are surgical, middle aged, and have fewer comorbidities.

Introduction

Due to recent health care modernisation, health care providers are becoming increasingly dependent on electronic medical records (EMR). Furthermore, the EMR has become a widely accepted source of data for epidemiological, health services, drug utilization, regulatory and safety surveillance research [1]. An EMR accumulates relevant information from a patient’s visit to a health care facility, including course of observation, treatment and discharge plan. Clinical documentation in the EMR can therefore exist in a variety of formats, ranging from free-text documentation to imaging files and numerical bedside monitor trends. With the use of machine learning techniques such as natural language processing and data mining, this complex EMR data can be analyzed to detect patterns in patient health status and healthcare delivery, which can in turn support clinical decision making [2].

Inpatient EMR documentation is also used to generate coded data in several countries [3]. To inform health policies and improve the delivery of health services, health care providers use inpatient coded data. The inpatient database relies heavily on electronic inpatient documents (specifically the discharge summaries). When the discharge summary (DS) is unavailable electronically, trained coders use the paper DS, which often presents challenges in discerning illegible handwriting for the presence of codable diagnoses [4]. As a result, many healthcare facilities are transitioning to a fully electronic-based documentation system [5].

The DS is a required document for any inpatient who is being discharged from the hospital. Specific to the province of Alberta, the College of Physicians and Surgeons of Alberta has a guideline which outlines the responsibility of the physician in preparing the DS [6]. The DS captures extensive information regarding a patient’s hospital visit, such as: care processes, courses of treatment, diagnoses lists, healthcare providers, and admission and discharge dates. A sample of a discharge summary is provided in Figure 1.

Figure 1: Template for Discharge Summary

Not only is this document essential in ensuring high quality databases, it is also the primary means of communication and collaboration between healthcare providers [7]. The quality and availability of the DS have been shown to have implications on risk of re-admission [8] and patient safety [9].

A document that is commonly seen as a substitute for the DS is the “Discharge Instructions”, an instructional carbon-copy document for the patient (commonly given post-surgery) which outlines the precautionary measures to avoid complications to the surgical site. However, it does not contain the same clinical information as the DS, and is seldom sufficient as a substitute for the DS, both in the creation of databases and the communication between healthcare providers. Therefore, the DS is arguably one of the most crucial documents in the EMR. Consequently, this study aimed to assess the completeness of the EMR through the presence of the DS, as well as to analyze the possible indicators for the presence of the DS.

Methods

Study setting

The Calgary Health Zone is one of the largest fully integrated, publicly funded, single-payer healthcare systems in Canada. It provides all medical and surgical care to residents of Calgary and surrounding communities in Southern Alberta. Contained in the region are 13 academic and community hospitals. Calgary, Alberta has a population of 1.5 million. In 2016, 70.2% of Calgarians were between the ages of 15 and 64. The average life expectancy in Calgary is 82.9 years. There are 4 large hospitals in Calgary, 3 of which were used in this study. All 3 hospitals have inpatient, intensive care and short-stay beds. The Foothills Medical Centre is an urban teaching hospital with a Level 1 trauma centre and 766 beds. It provides care to over 2 million people from Southern Alberta. The Peter Lougheed Hospital is a 600-bed urban teaching hospital. It serves the northeastern part of the city, which has a high influx of immigrants and lower socioeconomic status Calgarians. Lastly, the Rockyview General Hospital is a 650-bed urban community hospital which provides medical, surgical, and psychiatric services to Calgary and Southern Alberta. The fourth hospital, South Health Campus is an urban Calgary hospital that had recently opened upon commencement of the chart review, and was therefore not included in the study. All Calgary hospitals are administered by Alberta Health Services, the single health authority in the province of Alberta.

Study cohort

In this retrospective study, six registered nurses performed a chart review of 3,045 randomly selected inpatient charts (one chart for one patient) at three acute care hospitals in Calgary. A random sample of discharges were selected after a computer-generated random number was assigned to each record occurring between January 1st and June 30th of 2015. The nurses had varying clinical experience, both in years of practice and area of expertise. The intent of the chart review was to identify 50 of a derived set of Charlson and Elixhauser health conditions. The chart review took place between August 2016 and June 2017. Patients met inclusion criteria if they were adults (≥ 18 years), had an Alberta personal health care number, and had an inpatient visit for any service outside of obstetrics between January 1st, 2015 and June 30th, 2015. Obstetric patients are typically healthy with minimal comorbidities and a short length of stay. This population does not accurately represent the majority of inpatient hospital admissions and was therefore excluded. A final number of 3,011 patient charts were used for this study. Initially, 1,015 charts were included from each hospital. This number was determined using expert opinion on sample size needed for chart review analyses. Originally the data used in this study was meant to assess the difference in sensitivity of condition identification between two cohorts (ICD-10 and ICD-11). ICD-10 and ICD-11 data were made available, and chart review was done using both ICD-10 and ICD-11 coding standards. To assess the improvement of the ICD-11 compared with the ICD-10-CA data for each condition examined, the sensitivity of ICD-11 data relative to chart data with the sensitivity of ICD-10-CA relative to chart data was compared. Sensitivity is the binary proportion of cases with the condition identified in ICD-11 (or ICD-10-CA) data when the condition is present in chart data. We considered an absolute difference in sensitivity of 10% to indicate a meaningful difference in the improvement of data quality. Based on previous study findings of sensitivities and prevalence of conditions in the sample of the ICD-10-CA data, 3,000 chart reviews were needed to test the 10% difference in sensitivity of common conditions. From the initial 3,045, 30 were in the field of obstetrics and thus were excluded. An additional 4 patient charts were missing information from the chart review and were excluded (Figure 2).

Figure 2: Flow diagram of final cohort used for chart review

A chart was considered to have “missing information” if basic information routinely found on all patients was missing from the chart review data (i.e. patient demographic information, admission date, sex, age). This occurred due to oversight from the chart reviewer.

Datasets used

The provincial electronic health record system for Alberta Health Services began in 1997, and has since been growing in complexity, as healthcare services continue to expand in electronic availability. Since 2006, the system used by Calgary healthcare providers for documentation in the inpatient electronic medical record is Sunrise Clinical ManagerTM (SCM)- EMR. However, only data collected since 2011 has been validated for research use [10]. SCM-EMR contains the electronic medical record, as well as laboratory and diagnostic imaging reporting. It includes free text (e.g., DS), structured data (laboratory data), and numerical bedside monitor trend data.

In this study, SCM-EMR was used to draw information on the 3,011 randomly selected patient charts used in the chart review. The nurses were given access to both SCM-EMR and the paper copies of the chart to complete the chart review. For every patient visit, the chart reviewer firstly reviewed the patient’s electronic chart, and then reviewed the paper chart for the presence of documentation not found electronically. All documents in both the paper and electronic chart could be consulted to abstract data on the 50 identified health conditions. For the purpose of a larger study with the same research group at the University of Calgary, the data produced by SCM-EMR on these 3,011 patients was analyzed. During the analysis, it was found that 893 of the 3,011 inpatient SCM-EMR charts were missing a DS.

Extended chart review

The location of the DS (paper or electronic) was not documented in the original chart review. Therefore, it was unclear if the missing DS frequency found in this analysis was simply a product of a missing paper DS, or if the DS was entirely missing from the chart. A random 10% sample (85 charts) was thus drawn from the 893 missing DS charts to assess for the presence of a DS in paper format, which would not have been detected in the SCM-EMR data. The same randomization process used on the larger cohort was used for the 10% sample. The 85 charts were assessed for information on the unit and department from which the patient was discharged, as well as the attending physician’s specialty.

Study variables

For those with a missing electronic DS, a list of possible associated variables was collected. These variables were compiled using expertise from physicians and nurses. Age was categorized according to the parameters used by the Centers for Disease Control and Prevention (18–44, 45–64, 65–80, >80). Length of stay parameters were drawn from the Discharge Abstract Database within the Canadian Institute for Health Informatics (CIHI). According to CIHI, the 2018 average length of stay for Albertan patients was 7.8 days [11]. Therefore, the four categories used to classify length of stay were deemed appropriate by study authors (≤3, 4–7, 8–11, ≥12 days). The discharge day of the week was used to assess whether the presence of a DS was influenced by the patient’s discharge falling on a weekday or weekend. Due to decreased hospital staff during weekends, efficiency and documentation can be affected. Study authors included the day of the week to observe whether decreased efficiency on weekends would result in the absence of a DS. The DS is crucial for interprofessional communication, particularly when a patient is being transferred from one facility to another. Therefore, disposition was included as a potential indicator for a missing DS.

In an effort to assess whether residents and medical students producing the DS contributed to the missing summaries, DS author and level of training was included as a variable of analysis. However, for those with a paper DS, the signature and level of training were often illegible. For the remaining cohort with no DS, author was not identified since no DS was authored. Therefore, this variable was not used in the analysis. Finally, the 17 Charlson comorbidities and Charlson Comorbidity Index (CCI) were included in the analysis for associated variables.

Statistical analysis

Study variables were reported as numbers and percentages for categorical variables. Chi-Square test was performed to assess differences of associated variables between missing DS and non-missing DS charts. Logistic regression was used to identify the variables that were independently associated with missing DS. Logistic regression results were reported using odds ratios (ORs) and 95% confidence intervals (95% CIs). Logistic regression was adjusted for the following variables: Disposition, Hospital, Length of Stay, Sex, Weekend/Weekday.

A p-value of <0.05 was used for statistical significance. Analyses were performed using Stat 14.0 (Stat Corp, College Station, TX, USA).

Results

By comparing the 893 missing DS patients with the remaining 2,118 patients with a DS, the univariate analyses showed that age, being a surgical patient, a Charlson Comorbidity Index (CCI) of ≤1, as well as patients with myocardial infarctions, congestive heart failure, cerebrovascular disease, dementia, chronic pulmonary disease, diabetes with complications, and renal disease were statistically significant when assessed for absence of DS (Table 1 and Table 2). The percentage of charts with a missing DS for those factors showing statistical significance were low, ranging from 8% to 20%.

Table 1: Frequencies of patient and hospital characteristics assessed for association with missing discharge summaries in the EMR

Variables	Category	Total (n=3011)	Missing (n=893)(%)	P-value
	18–44	619	185 (29.9)	<.0001
	45–64	1022	370 (36.2)
Age (years)	65–80	830	243 (29.3)
> 80	540	95 (17.6)
Sex	Female	1515	455 (30.0)	0.6502
Male	1496	438 (29.3)
	≤ 3	1120	349 (31.2)	0.3747
Length of stay (days)	4–7	1019	303 (29.7)
	8–11	336	90 (26.8)
	≥ 12	536	151 (28.2)
	Foothills Medical Centre	1012	313 (30.9)	0.4544
Hospital	Peter Lougheed Centre	998	296 (29.7)
	Rockyview General Hospital	1001	284 (28.4)
Surgical patient	No	1912	320 (16.7)	<.0001
	Yes	1099	573 (52.1)
Discharge day of the week	Weekday	2436	710 (29.1)	0.2057
	Weekend	575	183 (31.8)
	Died	65	17 (26.2)	0.4402
	Discharged home/home setting with support services	2608	784 (30.1)
Disposition	Transferred to a long term care facility	121	28 (23.1)
	Transferred to another facility providing inpatient hospital care	143	42 (29.4)
	Transferred to other	35	13 (37.1)
	Signed out (against medical advice)	39	9 (23.1)

Table 2: Frequencies of comorbidities assessed for association with missing discharge summaries in the EMR

Variables	Category	Total (n=3011)	Missing (n=893)(%)	P-value
CCI score	0	888	352 (39.6)	<.0001
	1	596	182 (32.7)
	≥2	1527	359 (23.5)
Myocardial Infarction	Yes	101	9 (8.9)	<.0001
Congestive Heart Failure	Yes	338	42 (12.4)	<.0001
Peripheral Vascular Disease	Yes	149	37 (24.8)	0.1859
Cerebrovascular Disease	Yes	357	62 (17.4)	<.0001
Dementia	Yes	171	17 (9.9)	<.0001
Chronic Pulmonary Disease	Yes	444	89 (20.0)	<.0001
Connective Tissue/ Rheumatic Disease	Yes	151	32 (21.2)	0.0194
Peptic Ulcer Disease	Yes	974	257 (26.4)	0.0066
Mild Liver Disease	Yes	233	51 (21.9)	0.0069
Diabetes without complications	Yes	577	143 (24.8)	0.0044
Diabetes with complications	Yes	317	64 (20.2)	<.0001
Paraplegia and Hemiplegia	Yes	51	6 (11.8)	0.0048
Renal Disease	Yes	429	87 (20.3)	<.0001
Cancer	Yes	429	151 (35.2)	0.0067
Moderate or Severe Liver Disease	Yes	23	2 (8.7)	0.0271
Metastatic Carcinoma	Yes	138	27 (19.6)	0.0079
AIDS/HIV*	Yes	11	2 (18.2)	0.5236

*Acquired immunodeficiency syndrome/Human immunodeficiency virus.

The multivariable logistic regression suggested that patient age, surgical patient, and CCI score were all found to be independently associated with missing discharge summaries (Table 3).

Table 3: Variables associated with missing discharge summaries in the EMR by multivariable logistic regression*

Variable	Category	Odds ratio (95% Confidence Limit)	P value
Age	18–44	Reference
	45–64	1.44 (1.13 to 1.83)	0.0035
	65–80	1.18 (0.91 to 1.55)	0.2138
	> 80	0.85 (0.61 to 1.17)	0.3105
CCI Score	0	Reference
	1	0.75 (0.59 to 0.96)	0.0239
	≥2	0.67 (0.54 to 0.83)	0.0002
Surgical patient	no	Reference
	yes	4.87 (4.09 to 5.8)	<.0001

*Adjusted for: Disposition, Hospital, Length of Stay, Sex, Weekend/Weekday.

Specifically, surgical patients had 4.8 times the odds of having a missing DS than non-surgical patients (OR: 4.87, 95% CI: 4.09–5.80). Compared with patients with a CCI of 0, patients with a higher CCI score (≥2) were less likely to be missing a DS (OR: 0.67, 95% CI: 0.54–0.83). Additionally, patients between the ages of 45–64 years were close to doubling the odds of having a missing DS than those <45 years (OR: 1.44, 95% CI: 1.13–1.83). Those aged >64 had non-statistically significant increased odds of having a missing DS. The sample size was not sufficiently large to conduct a multivariable logistic regression with each comorbidity.

Within the 10% sample of missing electronic DS charts (n=85), 60% (n=51) were also missing paper discharge summaries. From those charts missing both electronic and paper discharge summaries, 78% (n=40) had discharge instructions. All 40 of the patients with discharge instructions were discharged from a surgical unit. The remaining 11 patients with no electronic or paper DS or discharge instructions were discharged from non-surgical units (Table 4).

Table 4: Variables associated with missing electronic or paper discharge summaries from the extended chart review

		Category
		Missing electronic only	Missing electronic and paper
Variable	Non-surgical	1	11
	Surgical	29	40
	Total (n=85)	30	51

Discussion

The DS is a crucial document within the patient chart, and its absence can have detriments on the level of completion of the EHR. Consequently, patient outcomes [8, 9], interprofessional collaboration [7], and administrative data quality [4, 5] are all impacted by an incomplete EMR. Our objective was to assess for the presence of a DS in inpatient charts, as well as variables associated with missing discharge summaries. Alberta Health Services’ current EMR system has been in use for over a decade. It is one of the most validated electronic systems in Canada, and has great utility for interprofessional collaboration and secondary research purposes [10]. Despite its utility and the evidence supporting a fully electronic EMR system, our findings demonstrated that EMR incompleteness was associated with patient baseline and setting characteristics. Interestingly, physicians are relying on paper charting or simply not creating discharge summaries when patients are middle-aged, surgical and have fewer comorbidities. To our knowledge, this is a novel finding. No previous studies have found associations between the aforementioned patient characteristics and the absence of electronic discharge summaries. The findings of our study will caution researchers when using EMR data pertaining to these three patient characteristics. This discussion delineates possible explanations for these associations.

Upon further assessment of the 85-chart sample, 70% were from outpatient surgical procedures, specifically urological and orthopedic procedures. The one urology clinic and two orthopedic clinics in Calgary were contacted for further information on DS practices. Across all clinics, staff reported limited need for a DS post-procedure, as the surgeon performing the procedure is often the surgeon that will see the patient in follow-up at the clinic. Additionally, given the simple nature of many of these procedures, there is no perceived need for a copy to be made to the family physician. A DS paper template is provided for surgeons at one orthopedic surgery site, albeit not used. Discharge instructions are thus the only document created, with the sole purpose of informing the patient about post-procedure care. However, both orthopedic and urological clinics reported creating a DS in the context of a complex case, wherein a nurse practitioner creates the DS, not the surgeon performing the procedure. Although we were unable to look further into whether or not those surgical cases with a DS were indeed more complex, we can speculate that it is physician preference that dictates whether or not a DS will be created, unless there is a complex case.

The reported incentive in surgical departments to create a DS for complex cases can explain the association found in this study’s larger cohort between lower CCI scores and a missing DS. A lower CCI score indicates fewer or less severe comorbidities, which can simplify the procedure. A simple procedure would thus not qualify as “comple” and would not be considered for a DS. Outside of the surgical realm, one study focusing on general medicine found that patients being discharged with a lower CCI score had nearly double the time to dictation from point of discharge (measured in days), compared to those with a higher CCI score, despite low CCI patients having significantly shorter lengths of stay. This apparent lack of urgency to create a DS could be reflected throughout healthcare provider practices for patients with fewer or less severe comorbidities, leading to the absence of a DS in this cohort of patients. Interestingly, the length of the DS (words used) in patients with lower CCI scores did not differ from that of patients with higher CCI scores, which could represent the intrinsic need for communication between inpatient and outpatient physicians [16], regardless of the complexity of the case. Additionally, the associated risks for readmission increase for every three days gone without a DS once a patient is discharged [17]. Therefore, for patients where no DS is ever created, the posed health risks could be detrimental.

With regards to the positive association between those aged 45–64 and the absence of a DS, there is a lack of literature to support this discovery. However, it could be related to the decreased complexity of these patients, and consequent non-urgency for creation of a DS. Compared to those aged 65 and greater, disease prevalences for many of the Charlson comorbidities, as well as the most commonly reported diseases (e.g. hypertension, type 2 diabetes, coronary artery disease), are far less frequent. Therefore, the perceived urgency in reporting on a patient undergoing a simple procedure with few comorbidities could be less compared to the urgency for reporting on a complex patient. This could result in an absence of a DS.

Our finding of missing EMR discharge summaries in surgical departments aligns with existing literature, with exception to our finding of a complete absence in electronic DS [12]. Most studies report on the illegibility or scarcity of information within the surgical DS, while we found the document to be missing all together [13]. The lack of standardization in the creation of a DS by the surgeon performing the procedure calls for further investigation into department- or, unit- culture norms. Possibly, there have been no voiced concerns within the healthcare system to incentivize surgeons to create a DS. However, this is dubious, given the extensive national and international literature on both general practitioner and patient dissatisfaction with post-procedure information and outcomes [12-14]. There is a dearth in literature regarding surgeon perspectives on the necessity of a DS. When broadened to perspectives from other specialties, one study found physicians often do not feel sufficiently informed on the patient’s health history to create a DS. Additionally, avoidance of redundancy of patient information within the EMR is a primary deterrent for the creation of a DS [15]. More investigation is needed on the surgeon’s perceived nonnecessity of a DS, as a means to understand why surgical cases are often missing a DS within the EMR and paper record.

In addition to having negative consequences on patient health outcomes, the absence of a DS can also affect the use of EMR data for secondary purposes [18]. In the process of transforming patient data into a data repository, a coder’s data abstraction and the resulting data quality can be hindered with an absent DS. Within coding practices, the DS is treated as a gold-standard document for coders to abstract, verify, and validate their coding. When the DS is missing, coders must use additional physician documentation (e.g. the History and Physical) to abstract their necessary codes. However, without the DS, if there is inconsistent information across different documents, coders cannot verify their codes against their gold-standard and are forced to omit codes. This leads to a lack of rigor in their coding as well as missing data within the database the codes are entered in, which decreases the quality of data. This data quality is crucial given its use for national statistics on population diseases and morbidity. Advances in medicine and research will be hindered if the missing DS phenomenon and subsequent poor data quality continues.

Limitations

Since some of the documents for charts within Calgary medical facilities are still in paper format, this study cannot fully report on the quality of the EMR; rather, it reports on the completeness of the patient medical record. Nonetheless, regardless of paper or electronic documentation, there was still a significant number of records missing a DS entirely.

Within the 893 charts, there is a possibility that charts were captured in the missing DS cohort, when there was actually a paper DS present. Nonetheless, the lack of an electronic DS within the electronic medical record is still relevant. Given the predominantly electronic form of communication between hospitals and general practitioners in Alberta, the inconsistency in availability of documentation in one single location can delay processes for practitioners searching for important health information. Additionally, the delay in receiving a paper document versus quickly accessing the electronic document from a general practitioners office can hinder health delivery services.

A further limitation is that some conditions abstracted in the chart review are more prevalent than others. Therefore, statistical power is greater in those common comorbidities and can facilitate a smaller p-value. Our analysts chose to adhere to statistical significance rather than percentages when reporting analyses. However, one interesting finding to be noted is the cancer versus metastatic cancer missing DS prevalence. Though statistically insignificant, cancer had the highest percentage for missing DS of all listed comorbidities (35.2%). However, metastatic carcinoma had a lower percentage (19.6%) for missing DS. This could potentially be related to the selection criteria for cancer during the study. All cancers, including benign, were captured. Given our finding that less complex patients are of lower priority for physicians in DS completion, it would be fitting that less severe cancers (i.e. benign) would also be of less importance with regards to DS completion. Contrarily, metastatic carcinomas are more severe diseases. Thus, the DS for patients with metastatic cancer would be of higher priority for physicians.

Lastly, due to inadequate resources, our sample size for assessing the presence of paper DS was 10%. While we recognize this is a small sample size, the purpose of this sample was to confirm our finding that the majority of patient charts in surgical departments are lacking a discharge summary. All of our analyses were performed on the 893 charts, and we believe the study’s power was not affected by this small sample size.

Conclusion

In conclusion, our chart review found a large percentage of incomplete EMR data, as indicated by a missing DS. The absence of a DS within the EMR can present several issues for healthcare processes, including hindered communication between hospitals and general practitioners, heightened risk of readmissions, and poor usability of coded health data. Our study describes and identifies indicators of missing electronic discharge summaries. In an era of integrating and relying on the EMR for patient care and secondary data purposes, efforts should be focused on improving DS documentation among the three strongest indicators for a missing DS: surgical wards, middle age patients, and those with fewer comorbidities. The findings of our study will alert researchers to carefully conduct analyses using EMR data, taking into consideration the completeness of the EMR, and its influencing factors. Additionally, these findings will propel further investigation on the reasons for missing discharge summaries among these three groups of patients.