Accuracy of Adverse Drug Reaction Documentation upon Implementation of an Ambulatory Electronic Health Record System.

BACKGROUND: Detection, monitoring and treatment of adverse drug reactions (ADRs) are paramount to patient safety. The use of a comprehensive electronic health record (EHR) system has the potential to address inadequacies in ADR documentation and to facilitate ADR reporting to health agencies. However, effective methods to maintain the quality of documented ADRs within an EHR have not been well studied.
OBJECTIVE: To evaluate the accuracy and effectiveness of ADR documentation transfer throughout the implementation of a comprehensive EHR system.
METHODS: Retrospective analysis of ADR documentation at a tertiary care pediatric hospital between January 2013 and June 2014. ADRs documented in the newly implemented ambulatory EHR, pharmacy system and hybrid health record system were extracted. Documentation inconsistencies and processes for managing ADR documentation within the EHR were reviewed.
RESULTS: A total of 115 patients with 260 unique ADRs were identified. Only 155 (60 %) of the identified ADRs were found in the ambulatory EHR system. The remaining 105 ADRs (40 %) were missing from the EHR when it was compared with the other systems. Seventy-two patients (63 %) returned for a follow-up visit, and each had their ADR documentation reviewed in the ambulatory EHR. Following the visit, 44 % of these ambulatory EHR records still included incorrect information.
CONCLUSIONS: We identified discrepancies in ADR documentation within hospital systems, which need to be addressed as healthcare institutions transition to EHRs. Processes related to the transfer of ADR information into the EHR should be clearly defined. To improve the quality of ADR documentation, steps to force complete and continual ADR verification should be introduced at early stages of implementation of a new EHR, and all responsible providers should play a role.

Key Points

Introduction

Adverse drug reactions (ADRs), defined as an appreciably harmful or unpleasant reaction resulting from an intervention related to use of a medicinal product, are common in children, accounting for up to 8 % of emergency department visits and 2.9 % of hospital admissions [1-3]. Information on the general population regarding ADRs is readily available, but monitoring and reporting of these effects in children are still lagging. Children differ from adults in terms of their risks of particular ADRs, yet many of these events are equally preventable, hence detection, monitoring and treatment of ADRs are paramount to patient safety [2, 3]. Although it is common practice for providers to document ADRs, effective methods to maintain the quality of ADR documentation are not well established [4].

The use of an electronic health record (EHR) to detect, monitor and document ADRs has emerged as a promising complement to traditional documentation systems [5]. An EHR is a digital version of a patient’s paper chart, which contains real-time, patient-centered records. It gives healthcare providers (HCPs) the ability to document and access patient information in a prompt and secure manner. This can be especially important in making timely clinical decisions related to ADRs. Although most hospitals already have processes in place to document ADRs, they are still heavily reliant on paper charts or some form of hybrid health record (HHR) system—a combination of a paper chart and an EHR. Not only do paper-based systems have a higher risk of documentation errors, but also new regulations will soon require all Canadian healthcare institutions to have improved processes for documenting and reporting ADRs [6]. Under the new Protecting Canadians from Unsafe Drugs Act (Vanessa’s Law), healthcare institutions in Canada will soon be required to report all serious ADRs [7]. This legislation strengthens the need for an effective ADR documentation and reporting solution. Uncovering the root causes related to ADR documentation discrepancies within a hospital sets the stage for identifying the steps to improve ADR reporting and quality. Ultimately, this will translate into better quality and patient safety by providing accurate and accessible ADR information needed to make the best clinical decisions.

As part of the initial phase of a hospital-wide, comprehensive EHR platform, the newly implemented ambulatory EHR system at our institution was designed for all clinical information documentation and retrieval, including documentation of ADRs and prescriber order entries. The next phase of implementation will add inpatient documentation, medication dispensing records and documentation of medication administration to the patient. Once complete, the EHR will be the primary patient record.

Prior to the introduction of the EHR, recording and reporting of ADRs was primarily performed in an HHR system, which included paper charts combined with an inpatient EHR containing only nursing and allied health documentation. Upon implementation of the ambulatory EHR, the plan for the import of ADR data and transfer into the new system involved data abstraction from the HHR system, followed by validation by a responsible HCP with the patient/family. While chart abstraction is a common procedure that hospitals use for transfer of information into an EHR, abstraction accuracy and quality need to be carefully monitored [8].

In the interim phase, at each outpatient visit, a clinician will review and document any ADR information in the EHR. At each inpatient visit, a clinician will initially review and document any ADR information in the inpatient EHR, which will then be confirmed by pharmacy staff during a medication reconciliation process with the patient and with the EHR, and any discrepancies will be updated. In principle, the combined methods should provide high-quality data, as the initial abstraction retrieves relevant ADR information, while the clinical review corrects inconsistencies.

As more hospitals transition to an integrated EHR solution for documenting all medical information, it is important that we regularly evaluate the process to ensure the accuracy and quality of ADR documentation. The purpose of this study was to evaluate the accuracy of ADR information transfer and the effectiveness of the newly implemented EHR system for ADR documentation upkeep.

Methods

Hospital and ADR Documentation Systems

This study was performed at a tertiary care pediatric hospital in Canada providing level 1 trauma and level 3 neonatal care services, with 6000 admissions and 200,000 outpatient visits each year. The referral base is approximately 2 million people within 5000 square kilometers in the immediate area and another 29,000 people within 2 million square kilometers remotely. In the ambulatory setting, there are 79 clinics and 40 services. The implementation of the EHR is occurring over several phases, starting with ambulatory clinics in October 2013 and completion in 2018.

The accuracy and consistency of ADR documentation between the hospital’s primary documentation systems were evaluated by comparing the number and type of ADRs found within each system. ADR documentation for a select cohort of patients was retrieved from three individual health record documentation systems: (1) the HHR system, which included paper charts combined with an inpatient EHR containing only nursing and allied health documentation (Sunrise Clinical Manager Version 5.0; AllScripts Healthcare Solutions, Inc., Chicago, IL, USA); (2) the pharmacy system (Centricity Pharmacy Version 9.0; GE Healthcare, Little Chalfont, UK); and (3) the ambulatory EHR system (EpicCare Ambulatory 2012; Epic Systems Inc., Verona, WI, USA). The ADRs found in the newly implemented ambulatory EHR system were then compared with the documentation from the other two systems. ADR inconsistencies were identified, and the processes for managing ADR documentation within the EHR were reviewed. This included reviewing the process related to transfer of ADR documentation from the HHR into the ambulatory EHR system and the process of ADR documentation quality verification by HCPs.

Population Selection and Inclusion Criteria

Patients with ADR documentation were identified as follows. A list of patients who had their past medical information abstracted into the ambulatory EHR system from January 2013 to June 2014 was obtained. During the ambulatory EHR implementation, patients had their information abstracted from the HHR and entered into the EHR. Abstraction was defined as retrieval and transfer of medical information (including information on ADRs) by trained health records personnel from the HHR system into the new EHR system. Every instance of an abstraction was documented as an abstraction encounter within the patient’s EHR. Because the ambulatory EHR system had been started only in October 2013, at the onset of this study in June 2014 the number of records with completed medical abstractions was relatively small in comparison with the number of outpatient visits.

In an attempt to exclude patients with no documented ADRs, this list was further refined to include only patients previously admitted to the hospital with ADRs documented in the pharmacy system. The study relied on the accuracy of ADR documentation in the pharmacy system to identify patients. This provided a final cohort of patients consisting of known documented ADRs, allowing for effective process evaluation of ADR documentation transfer into the EHR.

Collection and Comparison of ADR Documentation

ADR information listed in the pharmacy and EHR systems was extracted electronically, while ADR information in the HHR system was retrieved through manual data collection by a pharmacy research assistant from June 2014 to September 2014. An ADR was recorded if there was documentation suggesting a link between a suspected drug and a reaction. Details surrounding the time and duration of an ADR were recorded, and the ADR category was classified using the National Cancer Institute’s Common Terminology Criteria for Adverse Events (CTCAE) modified to specifically capture ADR details. The severity and grade of an ADR was recorded using the Merck Manual Classification of Adverse Drug Reactions [9, 10]. The location of the ADR documentation within the HHR system was also recorded.

Data were collected into a Research Electronic Data Capture (REDCap; Vanderbilt University, Nashville, TN, USA) database and analyzed in REDCap and Microsoft Excel (Microsoft Inc.) [11]. ADRs that were identified in the HHR or the pharmacy system but not in the EHR were identified. Descriptive statistics were performed by a senior pharmacy research assistant and hospital pharmacists. The frequency, counts and distribution of ADRs between the different documentation systems were compiled and compared (see Supplementary Fig. 1 in the Electronic Supplementary Material).

Quality Assurance

As the hospital’s standard of practice, all responsible HCPs had been instructed to verify ADRs abstracted in the EHR during subsequent patient visits. The completeness, quality and accuracy of this verification process were reviewed. ADR documentation in the EHR was considered accurate and consistent if it matched the documentation in the HHR and pharmacy systems. As soon as they were identified, immediate resolutions were taken to contact the HCPs and rectify unverified or discrepant ADRs that had the potential risk to cause serious patient harm or jeopardize the provision of care.

Results

At the time of the investigation, there was a list of 1562 patients for whom medical abstractions in the EHR were completed. After cross-referencing of this list with the pharmacy system to identify records with previously documented ADRs, 115 patients (7 %) fitted the inclusion criteria for the study.

ADR Documentation Quality

Patient records were reviewed for ADRs in each of the three ADR documentation systems. A total of 260 unique ADRs were identified; however, only 186 contained descriptive details allowing for grading of severity. Of these, 116 were graded as a moderate-severity reaction, while 12 were graded as severe or life threatening (Table 1).

Table 1

Severity of documented adverse drug reactions (ADRs)

Grade	Number of ADRs
1: mild	58
2: moderate	116
3: severe	11
4: life threatening	1

ADRs were graded according to the descriptive documentation available (n = 186)

Similarly, ADRs were classified according to the descriptive details surrounding a reaction; however, only 101 ADRs had sufficient information suitable for classification into categories (Table 2). The remaining ADRs were linked to a suspected drug but could not be classified on the basis of the documentation provided (e.g., a drug reaction to sulfa). Antibiotic-derivative ADRs were the most well documented therapeutic category, but details related to the type of reaction were often absent (see Supplementary Table 1).

Table 2

Classification of the types of documented adverse drug reactions (ADRs)

ADR	Count
Abdominal pain	1
Agitation	3
Anaphylaxis	4
Anorexia	2
Anxiety	1
Delirium	1
Diarrhea	3
Dyspnea	1
Edema face	1
Erythema multiforme	1
Erythroderma	2
Extrapyramidal disorder	1
Fever	1
Gastrointestinal pain	1
General disorders and administration site conditions: other	1
Hypoglycemia	2
Injection-site reaction	1
Insomnia	1
Investigations: other	1
Maculopapular rash	34
Musculoskeletal and connective tissue disorders: other	1
Nausea	1
Nervous system disorders: other	1
Pruritus	3
Psychiatric disorders: other	2
Renal and urinary disorders: other	2
Reproductive system and breast disorders: other	1
Respiratory failure	1
Respiratory, thoracic and mediastinal disorders: other	1
Seizure	1
Skin and subcutaneous tissue disorders: other	8
Stomach pain	1
Tremor	2
Urinary retention	2
Urticaria	3
Vascular disorders: other	1
Vomiting	6
Weight gain	1
Grand total	101

159 of the 260 ADRs did not have sufficient information for an accurate classification (n = 101)

ADR Documentation Process in the EHR

Of the 260 identified ADRs, only 155 (60 %) were found in the ambulatory EHR system. The remaining 105 ADRs (40 %) were missing from the ambulatory EHR when it was compared with the HHR and pharmacy systems (Table 3). A review of the ADR abstraction process revealed that ADR information was abstracted directly from an ADR medication sheet found in the paper charts of the HHR system. The sheet was absent in 27 records (23.5 %), and many ADRs were identified only from other locations within the HHR system (Fig. 1).

Table 3

Adverse drug reactions (ADRs) identified within the hospital’s three primary documentation systems

Total number of ADRs identified throughout all systems	Number of ADRs identified in the EHR	Number of ADRs missing from the EHR	Distribution of missing ADRs
260	155	105	64 ADRs documented in the HHR system but not in the EHR
			12 ADRs documented in the pharmacy system but not in the EHR
			29 ADRs documented in the HHR and pharmacy systems but not in the EHR

n = 115 patient records

EHR electronic health record, HHR hybrid health record

Fig. 1

Locations of adverse drug reaction (ADR) documentation identified within the hybrid health record (HHR) system; 235 ADRs were documented in the HHR, and many ADRs were documented in more than one location within the system (n = 115 patient records)

To maintain the quality and accuracy of ADR information, the ambulatory EHR was integrated with a function to review previously reported ADRs within the system itself. HCPs were prompted to verify ADRs with the patient or family during every visit and were encouraged to correct inconsistencies when appropriate. At the time of the analyses, 15 months post-EHR implementation, 72 (63 %) of the 115 patients in the cohort had returned for a follow-up visit and would have consequently had their ADR documentation reviewed in the ambulatory EHR. The reviews in the ambulatory EHR were performed primarily by nurses (47 %) and physicians (14 %), and infrequently by pharmacists (1 %) and social workers (1 %) (Fig. 2). The reviews resulted in updated or more accurate ADR documentation in 56 % of the records, yet the remaining 44 % of the ambulatory EHR records still had incorrect ADR information when these data was compared with those in the other two documentation systems. Nurses were involved in these incomplete ADR reviews 83.3 % of the time, followed by physicians (12.5 %) and social workers (4.2 %).

Fig. 2

Review of adverse drug reaction documentation in the electronic health record (EHR) system by healthcare providers during follow-up patient visits to the hospital. The reviews were performed over a period of 15 months post-implementation of the EHR system (n = 72 patient records)

Discussion

In most hospitals, methods to document ADRs include the use of traditional medical charts, HHR systems, and individual department-specific documentation systems, such as those used in the pharmacy or the emergency department. The system of choice depends on many factors, including the location of the ADR discovery and the time of ADR reporting, as well as the profession of the HCP involved. ADRs are often documented only in the system that is convenient and familiar to the HCP. Because of lack of integration between many traditional documentation systems, ADR information may be inaccessible to providers utilizing other documentation systems within the hospital. This results in inadequate, duplicated and missing ADR information, which can jeopardize the quality of care and patient safety.

Comprehensive EHRs provide a modern solution in complementing and improving the inefficiencies that exist with current ADR documentation methods. With correct oversight, EHRs can help institutions to better address limitations in ADR documentation and help to regulate ADR reporting to governing agencies. However, a successful EHR system must have both the capacity to accurately document ADRs and the processes needed to ensure that the quality of ADR documentation is maintained.

The integrated ambulatory EHR system was expected to decrease repetition and correct inconsistencies in ADR documentation. While this is expected to improve over time, the current processes were not effective in ensuring the quality of ADR information entered into the new EHR. A significant number of ADRs documented within both the pharmacy and HHR systems were absent in the ambulatory EHR. Considering the severity of many of the ADRs, these missing documentations can have significant impacts on patient safety, which emphasizes the need for processes to improve the quality of ADR documentation.

One factor that might have contributed to this discrepancy is that the ambulatory EHR is the first clinical system of an enterprise-wide, comprehensive EHR—which will include all ambulatory, acute care (inpatient care, operating rooms and the emergency department) and pharmacy systems—to be implemented. Even so, 15 months into the implementation of the EHR, only 60 % of the known extracted ADRs were captured in the system. This percentage was much lower than expected and was likely due to the fact that ADR information was pulled exclusively from a single ADR medication sheet instead of being abstracted from the entire chart or other systems. Prior to the implementation of the ambulatory EHR, the ADR medication sheet was the standard source for documenting and updating all ADRs. Although it was designed for this purpose, the functionality and reliability of the sheet was questionable. Not only was the ADR medication sheet often unavailable, but also no standards were in place for auditing this process. A significant number of ADRs found within the discharge summaries, physician letters and other sections of the charts were never listed on the ADR medication sheets, which led to incomplete ADR abstraction.

In this instance, the resources allocated to abstraction (retrieval from a defined section of the HHR) proved to be insufficient for retrieving ADR documentation. It is expected that complete abstraction of ADR information from all sections of the HHR system would have provided better results; however, this process is long, laborious and expensive. A better strategy would be to perform a pre-scope into the charts to identify specific sections of the HHR system that contained a high prevalence of ADRs. Expanding the ADR abstraction procedure to these other areas within the HHR system would likely have increased the quality of abstraction without significantly increasing the duration of the process. Furthermore, abstraction from other department-specific documentation systems, such as those used in the emergency department and pharmacy, would allow more opportunities to identify concise ADR information. The challenge remains in finding an optimal balance of time and resources for facilitating this process within an institution.

Although other healthcare institutions may have different systems for documenting ADRs, our results suggest that standardization for ADR validation and monitoring is currently lacking. As was demonstrated in this study, ADR documentation may be distributed in different locations throughout the hospital, making it difficult to retrieve accurate information in a timely manner. Devising structured means to make it easy and convenient to regularly review and validate ADRs, regardless of the system, may aid in improving the quality of ADR documentation [12].

To address the common problems of abstraction quality, the hospital applied a patient-centric strategy in an attempt to correct ADR documentation inconsistencies incurred in the abstraction process [13, 14]. This strategy involved using HCPs to provide continual oversight of ADR documentation quality by encouraging them to verify ADR documentation in the ambulatory EHR during clinic visits or hospital admissions. At the time of the analyses, 37 % of the patient cohort had not returned for a follow-up visit and did not have their ADRs reviewed. Nonetheless, for the patients who had their ADRs reviewed, a third of the EHRs had discrepant ADR documentation that was uncorrected or inaccurate. A possible explanation for the uncorrected documentation following a patient visit may be shortcomings in communication between the provider and patients. Concentrating on the primary purpose for the clinic visit, HCPs may not have taken the time to ask direct questions to fully verify the ADR documentation; likewise, patients and parents may not have the knowledge to communicate the details of a past ADR. Furthermore, there may be a difference between HCPs’ and patients’ understanding of the definition of an ADR. Typically, the question that is asked is whether or not a patient has “allergies” to medications, which may not elicit recall of ADRs. Similarly, HCPs in specialty clinics may not update information on ADRs to drugs used outside their specialty, because of lack of knowledge.

Because the patient–provider interaction is typically spent with nurses or physicians, pharmacists were involved in only 1 % of the ADR reviews. This low value was also likely a result of pharmacists updating ADR information (if ADRs were identified in the inpatient setting) within the pharmacy’s independent system and not in the new ambulatory EHR. Because our methodology measured ADR reviews only in outpatient encounters, ADR reviews performed by pharmacists for inpatients would not have been captured. Until synchronization of the pharmacy system and the EHR occurs, pharmacy staff have been encouraged to document ADRs in both systems, but the compliance is poor. In the interim phase, a new ADR that is identified in the inpatient setting would be documented in the patient’s EHR record only upon review at a subsequent outpatient visit. Once the EHR is fully implemented, all ADRs will be documented in the EHR and will be viewable by all, regardless of the type of visit. Allowing opportunities for pharmacists to provide input during ADR reporting at clinic visits would likely increase the accuracy of the ADR information [15, 16]. Maintaining accurate ADR documentation is the responsibility of all HCPs, and it is important that they work collaboratively to achieve this goal.

Oversight of the quality of ADR documentation has been lacking because of low awareness in maintaining ADR quality and the complacency of HCPs in accepting minimal standards for ADR reporting [17]. As new standards and legislation related to ADR reporting are expected, strategies to improve the quality and consistency of ADR documentation are necessary. While the EHR system at our hospital is still being implemented, it is most important to evaluate the effectiveness of ADR documentation and apply approaches to improve function. As with many technologies, successful implementation requires an appropriate amount of resources, time and commitment, and attention to the people, process and technology. End users must take the initiative to familiarize themselves with the important functions in a new EHR system and be engaged in providing process improvement feedback. Increasing awareness of ADRs throughout the hospital must then be followed by actions to simplify and encourage the process for accurate ADR review. For example, forcing functions can be used in the EHR to prevent HCPs from completing a request for patient discharge or billing unless they have first verified the listed ADRs. Fields within the reporting structure can also be restricted to a set of categorical choices so that consistency of the data can be maintained. Taking it one step further, algorithms can be developed to interpret signals or laboratory values that alert a HCP about the need to document an ADR or update existing ADR documentation [18, 19]. As more hospitals begin to transition to a comprehensive EHR system, effective methods for the transfer of ADR documentation and continual oversight of the quality of ADR documentation will be crucial to minimize adverse healthcare events.

This study was limited by its retrospective methodology and relatively small sample size. The small sample size was a result of the timing of the study in respect to the implementation of the EHR and was also due to the methodology used for selecting patients. The data collection period was 7–11 months into the EHR rollout, which did not allow much time for many complete patient abstractions. Furthermore, the method used to identify patients with known documented ADRs, using only the pharmacy system, was not very effective. This contributed to the small sample size, which was not fully representative of the entire hospital patient cohort. In the initial inclusion criteria, not all 1562 patients with complete medical abstractions in the EHR system were manually audited. Instead, the study relied on the accuracy of the pharmacy system to help identify patients. This was done in order to focus the efforts on evaluating the accuracy of documentation in patients with known ADRs, excluding those without ADRs. At the time, it was assumed that the pharmacy system was the most complete and accurate system for ADR documentation. However, it appears that the ADR information in the pharmacy system was not complete, as 78 (30 %) of the 260 ADRs (see Supplementary Fig. 1) were not actually recorded in the system. While relying on the pharmacy system to identify patients helped to narrow the criteria, this strategy excluded many potential patients with ADRs documented outside the pharmacy system, which were identified through another system. Consequently, the restricted inclusion criteria may have limited the findings, as they failed to provide a comprehensive evaluation of the process for ADR data transfer. For a comprehensive review, it is suggested that a list of patients with documented ADRs in both the HHR and pharmacy systems should be extracted. This would have increased the accuracy and eligibility of the patient cohort. With more resources, a review of the entire patient cohort with completed medical abstraction would be optimal.

Conclusion

Comprehensive integrated EHRs are expected to provide complete patient health histories and medication profiles in an accessible and centralized manner. These systems have the potential to address current inadequacies in ADR documentation and to facilitate ADR reporting to health agencies. Nonetheless, as healthcare institutions transition to EHRs for the better, discrepancies in ADR documentation within traditional hospital systems need to be addressed. Processes related to the abstraction and transfer of patient information into the EHR need to be well defined and developed. To further improve the quality of ADR documentation, steps to ensure more complete and continual ADR verification using an EHR system should be implemented, and all responsible HCPs should be involved in the discussion.

Below is the link to the electronic supplementary material.

Supplementary material 1 (DOCX 16 kb)

Supplementary material 2 (TIFF 161 kb)

Comprehensive electronic health records (EHRs) provide a modern solution in complementing and improving the inefficiencies that exist with current adverse drug reaction (ADR) documentation methods in hospitals.

The process for transferring ADR information between traditional paper-based and hybrid documentation systems into our new ambulatory EHR was scattered, laborious and inaccurate.

A successful EHR system must have the capacity to both accurately document ADRs and ensure that quality is maintained; hence, steps to ensure ongoing ADR verification using an EHR system should be implemented consistently, and all responsible providers need to be vigilant.