Assessing the impact of a quality improvement program on the quality and timeliness of discharge documents: A before and after study.

ABSTRACT: Whereas handover of pertinent information between hospital and primary care is necessary to ensure continuity of care and patient safety, both quality of content and timeliness of discharge summary need to be improved. This study aims to assess the impact of a quality improvement program on the quality and timeliness of the discharge summary/letter (DS/DL) in a University hospital with approximatively 40 clinical units using an Electronic medical record (EMR).A discharge documents (DD) quality improvement program including revision of the EMR, educational program, audit (using scoring of DD) and feedback with a ranking of clinical units, was conducted in our hospital between October 2016 and November 2018. Main outcome measures were the proportion of the DD given to the patient at discharge and the mean of the national score assessing the quality of the discharge documents (QDD score) with 95% confidence interval.Intermediate evaluation (2017) showed a significant improvement as the proportion of DD given to patients increased from 63% to 85% (P < .001) and mean QDD score rose from 41 (95%CI [36-46]) to 74/100 (95%CI [71-77]). In the final evaluation (2018), the proportion of DD given to the patient has reached 95% and the mean QDD score was 82/100 (95% CI [80-85]). The areas of the data for admission and discharge treatments remained the lowest level of compliance (44%).The involvement of doctors in the program and the challenge of participating units have fostered the improvement in the quality of the DD. However, the level of appropriation varied widely among clinical units and completeness of important information, such as discharge medications, remains in need of improvement.

Introduction

Handover of pertinent information between the hospital and primary care providers is necessary to ensure continuity of care and patient safety.[ More than half of preventable or ameliorable adverse events affecting patients after discharge from the hospital were linked to poor communication between the hospital caregivers and either the patient or the primary care physician.[ This communication relies on a discharge summary (DS) that is intended to be available soon after discharge. However, the literature shows that both the quality of the content and the timeliness of the DS could be improved.[ Regarding content, discharge medications are a recurring issue, when they are missing information or are incorrect in the DS.[ For instance, discharge medications were missing in 2% to 40% of the DS in a review conducted in 2006.[ Regarding timeliness of the DS, only 12% to 34% of the DS were available at the first post-discharge visit, negatively affecting the quality of care in approximately 25% of follow-up visits.[ A more recent review reported similar results with discharge medications reported in 56 to 100% of DS and only 67% of DS transferred to primary care providers within 48 hours.[

Interventions that would improve communication between hospital-based and primary care physicians at hospital discharge were identified based on 18 controlled studies.[ Use of standardized formats may be a way to improve the perceived quality of documents.[ Computer-generated DS and using patients as couriers appear as interventions that can shorten the delivery time of discharge communication.[ However, using patients as couriers raises the issue of pending results at discharge and the need for a two-step communication. A patient may be discharged with a discharge letter (DL) that is a provisional version of the final DS, but results of investigations and/or approval of senior doctors can be pending. This explains the need for a second document, the final DS.[ Another more recent systematic review shows that a structured approach to writing, educational training, and the use of a check-list are effective methods to improve the quality of the DS.[ Electronic DS are a way to standardize and potentially improve the quality of the DS, but they cover a large variety of situations depending on the existence or lack of an electronic medical record (EMR) and/or the possibility of automatically transferring preexisting patient information.[ For instance, DS completion by linking it to a preexisting patient information database has increased the proportion of DS completed within 48 hours from 45% to 58%.[ Trainees or junior doctors responsible for creating a DS may be unfamiliar with the format and the importance of discharge communication or have not been trained in DS writing.[ Educational interventions, with teaching sessions, audits and feedback, appear to have a positive impact on DS completion or DS quality, especially for junior doctors.[ Trainees or junior doctors responsible for creating a DS may be unfamiliar with the format and the importance of discharge communication or have not been trained in DS writing.[

Until recently, the content of a discharge summary (DS) was poorly defined in France.[ At the national level, the French National Authority for Health (Haute Autorité de Santé, HAS) defined in 2003 a quality indicator (QI) for the DS to be sent to the general practitioner (GP): it should be made available within 8 days following the discharge and include a minimum of 4 elements.[ This minimal DS that was sent within 8 days was found in only 53% of the medical records reviewed in the national assessment conducted on the 2015 data. Moreover, the content of these DS did not correspond to the expectations of the GPs who expressed the desire for a brief letter given to the patient at the time of discharge, and preferred a standardized and structured form.[ The French regulation has evolved and since January 2017 required delivery of a discharge letter (DL) to the patient at the time of hospital discharge, including 6 areas of administrative data and 6 of medical data (Table 1). French hospitals needed to improve the content in the discharge documents (DD) and to reduce their delay of production. A score measuring quality of DD (QDD score) was defined by the French National Authority for Health. The score ranges from 0 to 100 and hospitals need to reach a mean score of 80/100 to achieve the national performance objective. First public communication of hospital results regarding QDD score was planned in 2018.

Table 1

Details of criteria assessed in the Quality Discharge Document score (QDD score).

Type of data	Detailed criteria
Administrative data	1- Identity of patient including birth name, given name, date of birth and sex
	2- Delivery of the discharge letter to the patient
	3- Identity and contact details of the signing physician
	4- Identity and contact details of general practitioner (GP)
	5- Dates of admission and discharge
	6- Destination at discharge (home, rehabilitation…)
Medical data	1- Reason for hospitalization
	2- Significant findings from tests including test results pending at discharge and/ or procedures performed during hospitalization
	3- Admission and discharge medications, including for discharge medications: dose, frequency, formulation and route of each medication
	4- Synthesis of medical care and condition at discharge
	5- Follow-up plans
	6- Risks related to hospitalization: transfusion, multidrug-resistant bacteria

Based on an evaluation in our hospital and experimentations reported in the literature,[ a DS/DL quality improvement program was built and conducted. This program relied on the hypothesis that a better structure of the EMR completed in real time associated with the extraction of the pertinent data to automatically generate a DD, would result in both a higher quality of content and a reduced delay of production (reduction of physicians workload linked to medical documentation). The target was a mean QDD score of 80/100 within 2 years. The aim of this report is to describe the DS/DL quality improvement program set in a French university hospital (Paris) and to assess its impact on the quality and timeliness of discharge documents.

Methods

Setting

Our University hospital is part of the Paris hospital-network Assistance Publique – Hôpitaux de Paris. It includes 3 sites with 2 emergency services and is composed of acute units (medical, surgical, and obstetrical), psychiatric, and rehabilitation units. The hospital has 1397 beds, 818 senior physicians, and 291 residents are employed.

An EMR system is deployed and used in 90% of clinical units of our hospital. This EMR system is a software program designed to produce reports such as consultation reports or discharge summaries. It has been implemented in 2004, first used in a non-structured way (word processing templates) and later in a structured one allowing automatic data population of reports. Each clinical unit has 1 or 2 EMR referent(s) (1 medical secretary and 1 senior physician). The EMR is populated with administrative data regarding the patient (identity of patient, date of admission, date of discharge) and it is structured so that it allows data extraction and automatic generation of the DS. This EMR is composed of various structured forms (for instance the form related to entrance medical examination includes medical history and usual treatments) that are filled-in by junior and senior doctors all during the hospital stay. However, each clinical unit has developed templates with specific content, for medical documentation of the patient during hospitalization but also for the DS. Therefore, data contained in the DS are variable among the different clinical units.

Intervention

Several analyses were conducted to define the intervention. The process analysis of discharge document production confirmed the need to suppress intermediate and to complete in real time medical information in the EMR. A work group including 9 professionals from various clinical units of our hospital identified several brakes but also levers to improve quality of discharge documents. Levers identified by professionals were:

information and education of professionals regarding the meaning of information required in DL and how efficient use of EMR can reduce workload,

challenging units by providing feedback with a ranking of clinical units and

adaptation of the EMR to avoid redundant completion of data.

Other actions suggested but not retained in our program are summarized in a PICK chart (see supplementary material).

Finally, key success factors previously reported in literature for implementation of such a program included educational training with audit and feedback, standardization of the content, and integrating the DD into the EMR.[

A discharge document (DD) quality improvement program, including revision of the EMR and DS/DL templates, educational program, audit (using scoring of discharge documents) and feedback with ranking of clinical units, was conducted in our hospital between October 2016 and November 2018 (Fig. 1). The hospital Quality steering committee has commissioned the physician in charge of QIs and the EMR technical leader to lead this project. The project team includes the quality manager and the head of information technology of our hospitals, a project manager and a quality engineer.

Figure 1

Steps of the QI program conducted on quality and timeliness of discharge documents.

One of the issues regarding implementation of the DL was the need to ensure the inclusion in the patients EMR of a DL (i.e. a provisional DS) given to the patient and a definitive DS, achieved after discharge and sent to the GP for any pending test results. To address this problem, the EMR technical leader proposed creating in the EMR a DL template containing the information required, that is different from the DS and is populated with structured data from the EMR. This technical option of a DL different from the definitive DS was tested with a voluntary clinical unit. The structure of the whole EMR needed to be reviewed to ensure that all fields required were present, complete and sufficiently organized to be extracted for the automatic generation of the DL and also to reduce the physicians workload linked to medical documentation. The structure of the EMR and the DL of the pilot unit project were used as templates and as proof of the concept to involve the EMR referents of other clinical units.

An educational intervention, based on the test on the pilot unit, was conducted for the EMR referents of the other clinical units. The main objectives of the training were to identify the quality criteria of the discharge letter, to determine the optimal use of the EMR, and to be able to participate in the revision of the EMR to meet the quality criteria of the DL. After the training, each EMR referent met with the EMR technical leader to adapt the EMR together.

An information flyer has been created for residents and was distributed in May 2017 during the hospital welcome session. The flyer contained the list of DL quality criteria and an amusing example of a DL containing all the criteria.

Audits of the DS/DL using scoring (QDD score) were conducted during the intervention (intermediate evaluation) and after the intervention (final evaluation). The feedback provided after the audit was intended to promote awareness on the part of heads of the clinical units and to give EMR referents some keys for improvement. It included the ranking of services in descending order of the QDD score, detailed the reasons for non-conformity and suggested areas for improvement.

Main outcome measures

The impact of intervention on the quality and timeliness of discharge documents was measured by the proportion of discharge documents (DS or DL) given to the patient at discharge and the mean of a score measuring the quality of the content of the discharge document (QDD score). The national QDD score measures compliance regarding 12 criteria (6 medico-administrative criteria and 6 medical criteria – see Table 1) and it ranges from 0 to 100. One point is given for each compliant criterion. The score is 0 if the discharge document is produced after discharge or is not found in the EMR. The QDD score was presented in mean with 95% confidence interval (95%CI). For each criterion, Chi 2 test was used to compare proportions of compliant information observed before and after intervention. A P value of 5% or lower was considered to be statistically significant. Moreover, 2 process indicators were used to follow the project: the number of referents from clinical units educated on the quality of the discharge letter and criteria, the number of clinical units for which the EMR structure has been revised to meet quality criteria.

Design

The 2016 national mandatory evaluation was used as baseline evaluation for this project (160 medical records randomly selected from the second half of the 2015). This first evaluation of the QDD score showed a need for improvement and was used to build our DS/DL quality improvement program.

In the view of the Deming PDSA cycle, an intermediate evaluation of the QDD score was conducted between June and September 2017 to assess the impact of the improvement program on current practices regarding DS/DL (Fig. 1), and eventually to correct remaining issues (10 medical records per unit were reviewed).

The final evaluation was performed during the summer of 2018 on medical records from June 2018 and included 10 medical records per unit.

All evaluations were retrospectively conducted on the patients EMR. The patients were informed by posters and in the welcome booklet of their right to opt-out and may forbid the use of this data. Use of these data for evaluation received institutional review board approval from the French Data Protection Authority (authorization CNIL no.1320749v0). This procedure complies with the European General Data Protection Regulation, and files used for evaluations were declared to the AP-HP Data Protection Office (20190812110338).

Results

During January and February 2017, 67 persons, including one quarter of senior doctors, were educated in the importance of the quality of the discharge letter and in the associated quality criteria of the DL. Although EMR reviewing had been planned during the first half of 2017, by June 2017, one quarter of clinical units (11/41) had not yet engaged in reviewing their EMR. By September 2017, the structure of the EMR had been reviewed and modified to improve the quality of the DL for 95% of the targeted clinical units (38/41).

Evolution of the mean QDD score during the program is presented in Table 2 with detailed information on the proportion of information present/compliant for each criterion. The baseline evaluation showed a mean QDD score of 41/100 (95%CI [36-46]). This low score was first explained by the absence of a DD dated from the discharge day (found in only 64% of medical records reviewed). The lowest rates of present/compliant information were observed for admission and discharge medications (10%) and identity of the patient (33%). The intermediate evaluation showed a significant improvement as the proportion of DD given to patients had increased from 64% to 85% (P < .001), and the mean QDD score had also increased from 41 to 74/100 (95%CI [71-77]). Except for admission and discharge medications and for risks related to hospitalization, all other information had reached at least 80% of compliance in the DD. Risks related to hospitalization were significantly more often found in the DD (78% vs 50%, P < .001) and the proportion of required admission and discharge treatments that were included also increased in DD (10% to 26%, P < .001). Between intermediate and final evaluations, the proportion of DD given to patients had continued to increase from 85% to 95% (P < .001). The QDD scores also significantly increased to reach 82/100 (95%CI [80-85]) in June 2018. The proportion of compliant admission and discharge treatments reported also increased (26% to 44%, P < .001) but remained low compared to the level of conformity observed for other criteria.

Table 2

Evolution of Quality Discharge Document (QDD) score and conformity of data during the program for all clinical units.

	Baseline evaluation∗	Intermediate evaluation	Final evaluation
Observation period	Second half of 2015	June-September 2017	June 2018
Number of medical records reviewed	160	409	398
Number of discharge letters/discharge summaries (DL/DS) given to patient	102	346	377
% of DS/DL given to patient	63.8%	84.6%	94.7%
QDD score (mean [95%CI])	41 [36–46]	74 [71–77]	82 [80–85]
% of compliant data for each criterion † (n)
Administrative data
Identity of patient	33.3% (34)	93.4% (323)	85.4% (322)
Delivery to the patient	52.0% (53)	89.0% (308)	90.2% (340)
Identity and contact details of the signing physician	97.1% (99)	99.7% (345)	95.5% (360)
Identity and contact details of GP	78.4% (80)	90.8% (314)	84.1% (317)
Dates of admission and discharge	76.5% (78)	88.4% (306)	78.3% (295)
Destination at discharge	61.8% (63)	91.6% (317)	93.6% (353)
Medical data
Reason for hospitalization	90.2% (92)	99.7% (345)	99.7% (376)
Tests and/ or procedures performed	72.6% (74)	96.0% (332)	97.4% (367)
Admission and discharge medications	9.8% (10)	26.3% (91)	44.3% (167)
Synthesis of medical care and condition at discharge	61.8% (63)	96.8% (335)	92.6% (349)
Follow-up plans	86.7% (92)	95.7% (331)	93.1% (351)
Risks related to hospitalization	50.0% (51)	78.3% (271)	87.5% (330)

In 2018, the mean score per clinical unit ranged from 12/100 to 98/100. Table 3 presents the mean QDD scores relative to the type of clinical unit. Both rehabilitation and psychiatric units had mean QDD scores higher than 90/100.

Table 3

Quality Discharge Document (QDD) scores and length of stay observed in 2018 according to the type of clinical units.

	Number of medical records reviewed	Mean QDD score [95% CI]	Median length of stay [Q1-Q3]
Medical units	278	81 [78–84]	6 [3–14]
Surgical units	80	82 [76–88]	3 [2–6]
Obstetrical unit	10	87 [82–92]	3.5 [3–4]
Total MSO units	368	81 [79–84]	5 [2–12]
Rehabilitation units	20	93 [91–96]	40 [22–73.5]
Psychiatric unit	10	97 [94–100]	22 [13–34]
All clinical units	398	82 [80–85]	6 [3–14]

Discussion

The DS-DL quality improvement program conducted in our hospitals between October 2016 and November 2018 has increased the proportion of DD given to patients at discharge from 63% to 95%, and the mean QDD score went from 41/100 (95%CI [36-46]) to 82/100 (95%CI [80-85]). Thus, within 2 years, our hospitals have reached the national performance objective of a mean QDD score equal to 80/100. One of the key success factors for implementation of our quality program is the involvement of senior doctors in planning the intervention and in participating in the educational program and in revision of the EMR. Communication of the QDD scores, with ranking of services in descending order, has aroused the interest of heads of clinical units and challenges them. Some managers have asked for additional evaluations to track the impact of improvements undertaken in their unit.

Globally, the national objective for quality of discharge documents has been reached by our hospitals. However, the quality of DD remains relatively variable among clinical units (12-98/100) and appeared higher in rehabilitation and psychiatric units. This result is in line with a Dutch study that reported a variability in the rate of missing DL among hospital departments.[ Results regarding the QDD scores appeared better in units where a medical referent had been trained and was involved in the quality of discharge documents. Some, such as the psychiatric unit, were engaged in a continual process of improving quality by having regular DL reviews involving residents and conducting additional audits with feedback.[

Some results regarding the level of compliance need to be discussed. First, regarding administrative information, the very low level of compliance observed for identity of the patient during the baseline evaluation may appear surprising. This low level was mostly explained by the lack of birth name in patient EMR. At the time of baseline evaluation, only 1 name could be collected in the software used to manage patient identity without specification of the type of name (birth name or married name). Implementation of a new identity management system (in November 2016 for 2 sites and June 2017 for the third one) requiring both names has considerably improved completeness of patient identity information in the intermediate evaluation (33% to 93%). The 22% of missing or incorrect dates of admission and discharge observed in our final evaluation appears higher compared to a review that reported missing admission/discharge dates in 0 to7% of DS.[ But it is close to another study that reported missing or incorrect dates of hospitalization in 20 to 42% of DL.[ This discrepancy may be due to the level of requirement (missing vs missing or incorrect) and/or the type of document reviewed (DL vs DS). Identity and contact detail for the GP was missing in 16% of DD in our final evaluation. This result is consistent with previous findings that reported the name of primary care physician missing in 16% to 17% of the DS or DL.[ Delivery of discharge documents to the patients was already reported in 50% of DD during baseline evaluation, showing that medical practices had evolved before the change in French regulation.

Regarding medical information, in our final evaluation almost all criteria reached a level of compliance higher than 87%. Follow-up plans were lacking in only 7% of the DD, while previous studies reported an average of 14% missing or incomplete information in the DL and 30% to 58% in the DS.[ The quality of information regarding medications in our DD, with less than half of the DD containing compliant admission and discharge treatments (44%), needs upgrading. This low compliance rate may be explained by the level of requirement of the criterion: complete lists of admission and discharge medications including dose, frequency, formulation and route for all discharge medications. Consequently, this result is not comparable to previous studies that often focused on discharge medications and reported them lacking in 2% to 40% of discharge summaries. [ Using the UK National Prescribing Centre criteria for discharge medication (including doses, frequencies, routes of administration, formulations, and durations), a previous study reported non-conformity regarding discharge medications in 33% of audited DS and showed that deviations manifested particularly with medicine formulation and duration.[ The hospital electronic prescribing and medicines administration (HEPMA) system in our hospital is used only on 2 sites and this HEPMA system is not integrated into the EMR. Since there is no possibility for automatic transfer of information from the HEPMA to the DD, treatments are mostly transcribed by doctors in the EMR. As shown in an Australian study, implementation of an electronic DS system without HEPMA did not improve medication errors in DS since 12% of errors were found in handwritten summaries and 13% in electronic summaries, due to the common factor of transcription.[ In the UK, a before-after study following HEPMA implementation showed an improved quality of discharge documentation. There was an increase in allergy documentation and a statistically significant reduction in prescribing error.[ Fortunately, clinical information systems are evolving to allow automatic transfer of medication lists from electronic medication management systems to the electronic DS/DL.

Some limitations must be noticed. The impact of our program was assessed on the degree to which the DD was given to patients at discharge, and also on a score assessing the content of the discharge document (QDD), as defined by the French National Authority for Health (HAS). Assessment of the criteria included in the score corresponds to the presence or absence of information in the DD. The format and the qualitative content of DD (i.e., accuracy of information) were not assessed.[ Also, the summary length and the overall readability of the DD were not specifically assessed, contrary to some previous studies.[ The impact of our program on perceptions of GPs regarding DS/DL also was not considered. However, criteria included in the QDD score correspond to information requested in the DD by French regulation and are in agreement with key elements required by the GP in DD. Moreover, we used computerized DD that were integrated into the patients EMR. This corresponds to preferences of both patients and health care providers who preferred a computer-generated DS that has a structured format, a concise style, clarity and is time efficient.[ Finally 10 EMR per clinical unit were reviewed in each evaluation. This number may appear limited but corresponds to approximately 400 EMR for each evaluation. Moreover, each clinical unit corresponds to 1 specific organization: for instance internal medicine corresponds to 3 different clinical units according to their specialization.

Conclusion

This study shows the positive results of the DD quality improvement program conducted in our hospital since almost all patients are now given a DL/DS at discharge. Although global improvement of quality of DS/DL was observed, many issues remain, such as the quality of admission and discharge medications, the accuracy of information and the adequacy of the components of the DS/DL. In the future, implementation of a new EMR within the regional healthcare network is expected to improve the quality of DS/DL. Similarly, a larger involvement of the clinical managers in the quality programs may help to have a positive impact on safe transfer of care. Impact of this program on patient safety handover should be assessed including primary care physicians.

Acknowledgments

We want to thank all junior and senior physicians of our hospital for their daily involvement in quality medical care for patients, and especially the medical EMR referents who participated in the revision of the EMR.

We thank Pr Isaac Azancot, president of the patient medical record commission, and Franck Fouchere, Head of Information Technology, for their support in this quality improvement project.

We thank Alexandra Thomas, head of the medical secretaries of our hospital and the medical secretaries who participated in the audit of discharge documents during this project.

We also thank the Public Health department and especially Christelle Duteil for her help in audit organization and Dr Prescillia Piron for her advice on a previous version of this manuscript.

We thank Yannis Varni for his assistance in reviewing the technical language of this manuscript.

We would also like to thank Mary Alice Parsons for her skillful assistance in reviewing the language of this manuscript.

Author contributions

Conceptualization: Pénélope Troude, Raphaël Goudinoux, Fabien Martinez.

Data curation: Pénélope Troude.

Formal analysis: Pénélope Troude, Annie Brion, Raphaël Goudinoux, Fabien Martinez.

Investigation: Pénélope Troude, Isabel Nieto, Jean Laganier, Valérie Ducasse, Rémy Nizard.

Methodology: Pénélope Troude, Fabien Martinez.

Software: Annie Brion, Raphaël Goudinoux.

Supervision: Pénélope Troude, Fabien Martinez, Christophe Segouin.

Validation: Jean Laganier, Valérie Ducasse, Rémy Nizard, Christophe Segouin.

Writing – original draft: Pénélope Troude, Isabel Nieto, Annie Brion, Christophe Segouin.

Writing – review & editing: Pénélope Troude, Isabel Nieto, Annie Brion, Raphaël Goudinoux, Jean Laganier, Valérie Ducasse, Rémy Nizard, Fabien Martinez, Christophe Segouin.