Implementation of Computerized Physician Order Entry for Critical Patients in an Academic Emergency Department is Not Associated with a Change in Mortality Rate.

INTRODUCTION: There is limited literature on the effect of computerized physician order entry (CPOE) on mortality. The objective of our study was to determine if there was a change in mortality among critically ill patients presenting to the emergency department (ED) after the implementation of a CPOE system.
METHODS: This was a retrospective study of all critically ill patients in the ED during the year before and the year after CPOE implementation. The primary outcome measures were mortality in the ED, after admission, and overall. Secondary outcome measures included length of stay in the resuscitation area of the ED, length of hospital stay, and disposition following hospitalization. Patient disposition was used as a marker for neurologic function, and patients were grouped as either being discharged to home vs. nursing home, rehabilitation center, or a long-term healthcare facility. We analyzed data using descriptive statistics, chi- square, and Wilcoxon rank sum tests.
RESULTS: There were 2,974 critically ill patients in the year preceding CPOE and 2,969 patients in the year following CPOE implementation. There were no differences in mortality between the two groups in the ED, after admission, or overall. The pre- and post-CPOE mortality rate for the ED, hospital, or overall was 2.52% vs. 2.02% (P = 0.19, 95% confidence interval [CI] -0.3 to 1.3), 7.8% versus 8.29% (P = 0.61, 95% CI -1.9 to 0.9), and 10.32% vs. 10.31% (P = .60, 95% CI -1.5 to 1.6), respectively. There was no difference in hospital length of stay between pre- and post-CPOE patients (3 days versus 3 days), a difference of 0.05 days (95% CI -0.47 to 0.57). Length of stay in the ED resuscitation area was longer in the post-CPOE group (31 versus 32 minutes), a difference of -1.96 minutes (95% CI -3.4 to -0.53). More patients were discharged to home in the pre-CPOE group (66.8% versus 64.3%), a difference of 2.54% (95% CI 0.13% to 4.96%).
CONCLUSION: The implementation of CPOE was not associated with a change in mortality of critically ill ED patients, but was associated with a decrease in proportion of patients discharged to home after hospitalization.

INTRODUCTION

“To Err is Human: Building a Safer Health System,” a 1999 report by the Institute of Medicine, reported that medical errors contribute to between 44,000–98,000 deaths per year in the United States.1 Computerized physician order entry (CPOE), a specific component of electronic medical records (EMR), has been touted as an effective tool for decreasing medical errors. The LeapFrog Group, initially funded by the Business Roundtable and launched in 2000, includes CPOE as one of its 4 leaps in improving hospital quality, safety, and affordability.2 Several studies demonstrate a decrease in medication error rates, potential errors, and harmful adverse drug events when CPOE is used.3–5 In a recent study of 3,364 hospitals, CPOE hospitals outperformed non-CPOE hospitals in medication ordering quality-related measures.6 There has been an increasing belief that EMRs can improve the quality of patient care, decrease medical errors, and improve healthcare delivery efficiency. As a result, The American Recovery and Reinvestment Act of 2009 set aside approximately $17 billion for healthcare industry incentives to adopt EMR systems for use with Medicare and Medicaid patients.7

Only a few studies have examined mortality outcomes related to the introduction and use of CPOE. These studies have conflicting results. Han et al8 demonstrated an increase in mortality from 2.8% to 6.57% after the introduction of CPOE in a tertiary-level care pediatrics hospital using their critical care transport database to identify1,942 patients in the study. Their conclusion, an unexpected increase in mortality, might indicate that “surrogate outcome measures such as medication error rate or adverse drug events (ADEs) alone may not be sufficient to determine CPOE efficacy.” A potentially significant methodology issue in this study was the large difference in observation study periods between the preand post-CPOE groups. The pre-CPOE study period consisted of the preceding 13 months, while the immediate post-CPOE study period was 5 months. Sittig et al9 argued that the primary reason CPOE is prone to failure is because of the profound alteration in patient care workflow processes developed over many years and proven to work. Thus, the 5 months post CPOE was not a long enough period for physicians to adapt to their new routines and responsibilities. Alternatively, Del Beccaro et al10 found no increase in mortality rates in a 2,533-pediatric-intensive-care patient study after the introduction of CPOE. They concluded that differences in the implementation of CPOE between the 2 institutions were the primary factors affecting the differing mortality rates. Keene et al11 published a study involving 1,291 pediatric intensive care patients and demonstrated no difference in mortality associated with CPOE implementation. They also concluded that “careful preparation, unit by unit tailoring, and extensive technical support” may have been keys to their results. Only one study to date has demonstrated a decreased hospital-wide mortality rate after implementation of a CPOE.12 This study involved all admitted pediatric patients to a children’s hospital. A total of 80,063 pre-CPOE and 17,432 post-CPOE patients were studied, and the mean monthly adjusted mortality rate decreased by 20%.

The few studies that have looked at mortality pertain only to the limited population of admitted pediatric patients. Our goal was to determine if CPOE was associated with any change in mortality in our diverse critically ill emergency department (ED) patient population.

METHODS

This was a retrospective study of critically ill patients in the ED one year before and one year after implementation of an EMR that uses CPOE. The primary outcome measure was ED, in-hospital, and overall mortality. Secondary outcome measures were length of ED stay in the resuscitation area, length of hospital stay, and patient disposition following hospitalization. We used patient disposition as a gross marker of neurologic function, with discharge to home an indicator of largely independent function, and skilled nursing or rehabilitation center discharge as markers of impaired function.

This study was conducted in the ED of an urban county hospital with an annual patient census of 99,000 in 2008. Critically ill patients were defined as patients undergoing treatment in the stabilization area of the ED. Treatment in this area is at the discretion of the treating physician, and includes patients identified as having an immediate life threat, regardless of age, etiology of illness, or mechanism of trauma. The decision to treat a patient being transported by advanced life support (ALS) ambulances in the ED stabilization room was made by the transporting paramedic team using established protocols and conveyed to the treating physicians before patient arrival. Only ALS-transported patients deemed to be critically ill or injured were placed directly into the stabilization room.

All critically ill patients treated in the ED for one year before and one year after CPOE were included in the study. The post-CPOE phase started immediately after implementation. This was designed to examine the possible increase in mortality associated with lack of gross familiarity of the CPOE system, and associated changes in workflow. Acute respiratory distress, unstable vital signs, known myocardial infarction, penetrating trauma to the torso, blunt trauma from a significant mechanism of injury, marked alteration in level of consciousness, and suspicion of acute stroke are examples of patients treated in our ED stabilization area.

Physician documentation and CPOE were implemented in our ED using EpicCare Inpatient Clinical System ™ (Epic Systems Corporation, Verona, Wisconsin), on February 1, 2007. All other inpatient care areas of the hospital went live with CPOE on February 1, 2007 as well, but physician electronic documentation did not occur in these areas until August 2007. Prior to EpicCare, our ED used EmStat ™(Allscripts-Misys Healthcare Solutions, Chicago, Illinois), an EMR limited to use in the ED but without CPOE capability for critical care patients. EmStat and EpicCare were respectively turned off and on simultaneously.

One of 6 chart abstractors (4 physicians, 1 nurse, and 1 medical student) reviewed all critical care patient charts from these periods. The principal researcher trained each abstractor using a structured data abstraction form. Abstractors were not blinded to the study hypothesis. We did not perform inter-rater reliability testing. All but one study reviewer (representing 100 cases) abstracted pre-CPOE and post-CPOE charts. Data abstraction started in December 2008, 10 months after the last post-CPOE patient was managed in the stabilization room. Data was entered into a Microsoft Access ™ (Microsoft Corporation, Redmond, Washington, version 2003) database.

The primary end point was ED and hospital mortality. Secondary outcomes included time in the stabilization room, length of hospital stay, and place of disposition after discharge from the hospital. The length of stabilization room time was defined as the time from initial patient arrival to departure from the ED stabilization room. It does not include time spent on diagnostic tests outside of the stabilization room, such as computerized tomography, operating room time, or time spent in other parts of the ED.

Descriptive statistics were used as appropriate. We report 95% confidence intervals (CIs) around the differences between groups for continuous data. Differences in proportions were tested by X, and differences in medians were tested by Wilcoxon rank sum. We used an alpha probability of 0.05 as the threshold for statistical significance. All statistics were performed with Stata, version 10 (StataCorp, College Station, TX).

Description of Pre-CPOE and Post-CPOE Ordering Systems

Personnel staffing for stabilization room cases were the same pre-CPOE and post-CPOE, with a team consisting of an emergency medicine (EM) faculty, a third-year EM resident, a first-year resident, 2 nurses, a nursing assistant, and a respiratory therapist. A medical student, if available, recorded events on a standardized paper form. This form was not part of the patient’s medical record, but was utilized after case completion by the physicians and nurses for their medical charting.

Pre-CPOE Ordering System

All physician medication orders were given to the 2 nurses verbally. After the case was completed, the nurses would document the orders in the paper stabilization room medical record log, which the faculty physician would then review and sign. This would become part of the official medical record. Orders for laboratory studies or medical imaging were given verbally to the nursing assistant, who would complete paper ordering forms and send these forms to the appropriate departments. There was no systematic process to ensure verbal order patient safety, such as writing down and verbally repeating physician orders.

The vast majority of medications were stored in the stabilization room, and pharmacy was not involved with checking the orders. For medications stored only in pharmacy, the pharmacy was called by the nurse with the requested medication, which would be checked for appropriateness and accuracy before delivery to the stabilization room. The system for obtaining medications stored in the pharmacy post-CPOE remained the same, although the pharmacy could directly see the order.

Initial Post-CPOE Ordering System

Patients were not entered into the EMR until their physical arrival in the stabilization room, and were not pre-registered. Electronic placement of the patient was performed by either the nurse or nursing assistant after patient arrival. As a result, physicians could not place orders until patient arrival in the stabilization room. This resulted in a delay to CPOE of approximately 3–5 minutes.

The nurses and physicians for critical cases work within the confines of the stabilization room in close proximity. Although not formally studied, there did not appear to be major issues with delayed recognition of physician orders as a result of CPOE.

At the time of initial implementation of CPOE, order sets were available for the physicians. These order sets were specific for the patients’ clinical problem. (e.g. major trauma(blunt and penetrating combined), respiratory distress, cardiac arrest, acute myocardial infarction, stroke, or overdose). The order sets contained lists of the commonly prescribed medications, laboratory studies, and radiographic imaging. Clicking in a box for a medication would open another window which would require several clicks to specify the exact dosage, frequency, and method of medication delivery. Medication orders given on a per weight basis required electronic placement of the patients’ weight before such an order could be placed. Clicking on the requested radiographic studies would open up a window requiring additional clicks for information, such as indications for the study, pregnancy status of the patient, and whether or not the patient had a contrast allergy. Specific contrast orders were generated automatically by entering a contrast-enhanced computed tomography order itself.

All medication, laboratory, and radiographic imaging orders had at least 2 warning box statements requiring the physician to affirm their desire for the orders in question.

No laboratory or radiographic imaging would be performed without CPOE. The workflow for EM nurses required CPOE prior to administration of medications. Verbal orders were only permitted in cases requiring immediate life-saving medication administration, e.g., epinephrine for severe airway compromise from an allergic reaction. Following the verbal orders, physicians were still required to perform CPOE.

No definitive workflow processes were established dictating who would be entering the physician orders. It was policy that a physician needed to physically place the electronic orders. This was done by any of the physicians in the stabilization room, including faculty and any graduate level of resident.

The CPOE terminals were located immediately adjacent to the patients’ stabilization room gurney, and there were enough present for nurses, nursing assistants, and physicians to avoid sharing terminals.

There appeared to be a steep learning curve for physician order entry. Anecdotally it appeared that older EM physicians were more challenged than younger and presumably more computer-savvy physicians. A rough time estimate for physicians becoming acclimated to the electronic order entry was 1-2 weeks. An experienced user of the CPOE system would be able to place all orders for a given case in 5 to 10 minutes, depending on the number and complexity of the orders.

This study was deemed exempt by our Human Subjects Research Committee.

RESULTS

There were 2,974 patients in the year preceding CPOE and 2,969 patients in the year following CPOE implementation. No stabilization room cases were excluded from the study.

Table 1 shows the demographics and baseline presenting variables for the pre-CPOE and post-CPOE groups. There were statistically significant differences between the 2 groups for age, initial blood pressure, and initial respiratory rate. There were no differences in the 2 groups for gender, percent presenting in cardiac arrest, percent intubated, initial Glascow Coma Scales, or initial oxygen saturations. There was a slight decrease in the number of penetrating trauma patients in the post-CPOE group, 10.2% vs. 8.6% (−1.6, 95% CI −3.1 to −0.1).

Table 1.

Demographics and baseline clinical variables before and after implementation of computerized physician order entry (CPOE) (medians with interquartile range).

Category	Pre-CPOE	Post-CPOE	Difference (95% CI)
Total patients	2,974	2,969	−5
Age (median, QIR)	44, 27–59	46, 28–61,	1.15 (−2.2 to −0.6)
% Male	67.4	67.8	0.3 (−2.8 to −2.0)
% Presenting to ED in cardiac arrest	2.62	2.36	−0.23 (0.5 to1.1)
% Presenting to ED already intubated	11.37	10.64	−0.73 (−0.9 to 2.3)
% Intubated in ED	18.64	19.27	0.73 (−2.6 to 1.4
Initial systolic blood pressure in mmHg	133,113−150	135,115–154	2.78 (−4.7 to −0.8)
Initial heart rate	95, 80–112	95, 79–113	0.82 (−2.59 to 0.95)
Initial respiratory rate	18, 15–23	19, 16–23	−0.5 (−0.92 to −0.08)
Initial Glasgow Coma Scales	15, 10–15	15, 10–15	0.11 (−0.15 to 0.38)
Initial oxygen saturations	99, 96–100	99, 96–100	−0.31 (−0.84 to 0.21)

CI, confidence interval; ED, emergency department

Table 2 shows the place of disposition of patients from the stabilization room of the ED to the hospital. There was a slight increase in the number of patients discharged from the ED stabilization room to the computerized tomography and intensive care units in the post-CPOE group, and a slight decrease in the number of patients discharged to a hospital floor bed.

Table 2.

Disposition from the emergency department stabilization room.

Category	Pre-CPOE	Post-CPOE	Difference (95% CI)
Home (%)	0.40	0.13	0.27 (0.0087, 0.53)
Specialist(%)	0.40	0.54	−0.14 (−0.49, 0.21)
CCU (%)	2.05	2.76	−0.71 (−1.5, 0.069)
Morgue (%)	2.52	2.02	0.5 (−0.26, 1.3)
OR (%)	6.93	6.16	0.77 (−0.49, 2.0)
Floor (%)	30.06	26.78	3.3 (0.99, 5.6)
ICU/CT (%)	57.63	61.60	−4.0 (−6.5, −1.5)

CPOE, computerized physician order entry; CI, confidence interval; CCU, critical care unit; OR, operation room; ICU, intensive care unit; CT, computed tomography

Table 3 shows the primary and secondary outcome measure results. There was no statistical difference between the 2 groups in mortality rates in the ED, or at any time after admission. However, more patients were discharged from the hospital to places other than home in the post-CPOE group. The length of time patients were in the ED was statistically higher in the post-CPOE group, but the increase was not clinically important.

Table 3.

Primary and secondary outcome measures (Medians with interquartile ranges)

Category	Pre-CPOE	Post-CPOE	Differences (95% CI)
Died in ED (%)	2.52	2.02	−.5 (−0.3 to 1.3)
Died in hospital (%)	7.80	8.29	0.49 (−1.9 to 0.9)
Died in either ED or hospital (%)	10.32	10.31	−0.01 (−1.5 to 1.6)
Total ED stabilization room time in minutes (median, IQR)	31 (22 to 45)	32 (23 to 47)	−1.96 (−3.4 to −0.53)
Length of hospital stay in days (median, IQR)	3 (1 to 7)	3 (1 to 7)	0.05 (−0.47 to 0.57)
Discharged home versus all other dispositions (%)	66.8	64.3	−2.54 (−0.13 to−4.96)

CPOE, computerized physician order entry; CI, confidence interval; ED, emergency department; IQR, interquartile range

DISCUSSION

We found no change in mortality in the critically ill patients presenting to the ED in the year following CPOE implementation in the ED, after admission, or at any point during their stay in our hospital. To our knowledge, this is the first study examining mortality rates of all critically ill patients presenting to an ED before and after the introduction of CPOE. Demonstrating no change in mortality on first glance would appear to be a neutral or even positive result. However, one of the primary reasons for the institution of CPOE is to decrease medical errors that lead to significant morbidity and mortality. The “To Err is Human” report from the Institute of Medicine, demonstrating an additional 44,000 to 98,000 deaths per year due to medical mistakes, is frequently cited among the principal reasons for pursuing CPOE, and in a larger context, complete EMRs.

Our study found a significant decrease in the number of patients being discharged home from the hospital following CPOE implementation. This data was examined as a surrogate method to determine functional outcome. The inference is that patients discharged directly to home from the hospital likely had a better functional outcome than those patients discharged to either a rehabilitation or nursing home facility. Our data would seem to indicate that although a difference in mortality was not observed, the post-CPOE group fared worse as evidenced by less patients being discharged directly to home. Although the difference appears small, if extrapolated to all critically ill patients in the U.S., it represents a large number of discharges. In 2007 and 2008, 3.4% of our 196,291 ED patients were critically ill. In 2006, there were 119.2 million visits to EDs in the U.S. 13 Applying our 3.4% rate of critically ill patients to these 119.2 million patient visits nationally would grossly estimate 4 million critically ill patients cared for in EDs across the country. The discharge home rate after CPOE introduction demonstrated in our study, if applied to these 4 million patients, would estimate an additional 100,000 patients discharged to dispositions other than home.

There was an increase in the number of patients discharged from the ED stabilization room to the computerized tomography and intensive care units in the post-CPOE group, and a decrease in the number of patients admitted to a hospital floor bed. The etiology for these changes is unclear. No significant changes to our management of this patient population were made during the study period. The intensive care unit and computerized tomography unit disposition numbers were grouped together, as a detailed breakdown for each of these 2 units was not available. The post-CPOE increase in disposition to the intensive care unit and computerized tomography group might reflect an increase in use of computerized tomography, an increase in patients admitted to the intensive care unit, or both. Additionally, an initial decision to admit a patient to an intensive care bed might be changed to a floor bed if computerized tomography did not reveal significant illness or injury. This disposition change would occur in the computerized tomography suite, not in the ED, and is not reflected in our reported disposition data.

The post-CPOE group demonstrated a longer stabilization room time. This small difference is not clinically significant (−1.96 minutes). The added time for care of these patients might be attributable to increased or additional time needed for CPOE.

Although there were more self-reported nursing medication errors in the post-CPOE time period, this was not a defined data point included in our abstraction of charts.

Major Issues and Subsequent Improvements in Physician Order Entry

The most consistent complaint among EM faculty with implementation of physician order entry was the perception of time being taken away from the patient’s bedside in the critical first few minutes of stabilization room cases for the electronic placement of orders. Instead of focusing on the patient, at least one physician in the room was focused on a computer monitor with his “back to the patient.”

The second major complaint was simply related to the ease of use of the system. Physicians went from the easiest method of order entry, i.e., verbal, to a method that required numerous and seemingly extraneous mouse clicks and typing. The initial order design was for the physician to verify, with username and password, every medication order being given, despite already being logged into the EMR as the user of record. This was an initial safety feature designed to prevent another user from entering orders under the name of the signed-on user in the event he had left their terminal without closing the EMR.

A third major complaint involved the consequences of inadvertent lack of electronic order entry. Radiographic or laboratory studies would be delayed until the order was placed. Early in the post-CPOE time period, it was not uncommon for physicians to be waiting on the results of a study only to discover the study had not yet been performed due to lack of order placement.

A fourth major complaint was that the software was slow to respond to mouse clicks, screen changes, and the opening of new windows. This was the result of an insufficient number of servers, as well as the speed of the servers.

Since the initial implementation of CPOE, there have been numerous process improvements. Critical patients are now electronically pre-arrived prior to their physical arrival in the stabilization room, allowing for orders to be placed prior to actual patient arrival. A rough estimate is that 75% of all orders on critically ill or injured patients can be placed before patient arrival.

Order sets have greatly improved in terms of efficiency. Order sets have become more intuitive and specific, requiring fewer mouse clicks, screen changes, and pop-up windows. A general stabilization room order set, not specific to any individual clinical problem, was developed with pre-clicked checkboxes. This general stabilization room order set contained orders that would be placed on the majority of stabilization room cases, regardless of specific clinical presentations. Any unwanted tests, for example, pregnancy test in a male patient, need to be unchecked before finalizing the order set. Additionally, the requirement for physicians to verify each medication order with their username and password was abandoned.

A number of pre-determined critical medications, typically those used in advanced cardiac life support and rapid sequence intubation, are now handled in an expedited format with verbal orders. These medications were chosen because they are often ordered during a time in a critical care case where physicians need to focus on the patient and not be distracted by CPOE. The nurse administers the medication, and then immediately electronically documents this medication activity. Three separate things happen simultaneously when the nurse does this. First, a physician verbal order is electronically placed for the medication. Second, the electronic Medication Administration Record (MAR) is notified that this medication has been administered. Third, an electronic verbal order message is sent to the physician’s In-Basket within the electronic health record. The In-Basket has items requiring physician action on charts, such as documentation, billing, or signing of verbal orders. This verbal order message requires electronic signing before it can be removed from the physician’s In-Basket. This can be accomplished after patient care is finished. The number and speed of the servers was addressed and corrected, with improvement in the system.

LIMITATIONS

This is a retrospective chart review, and examines a site specific hypothesis regarding the effects of CPOE implementation. Our experience may not be generalized to other healthcare institutions.

We did not analyze or test for inter-rater reliability, which could result in data collection differences. However, the primary endpoint of stabilization room or hospital death is straightforward. What could be affected by inter-rater variability is the degree to which the 2 patient populations are judged to be similar.

Our ED had been using an EMR without CPOE for critical cases prior to the switch to EpicCare Inpatient Clinical System™ (Epic Systems Corporation, Verona, Wisconsin). It is possible that the transition to CPOE for critical cases in the ED was not as difficult given prior experience. However, CPOE and electronic charting were both new to the in-house physicians.

There were differences in age, initial blood pressure, and initial respiratory rate between the 2 groups. These were judged to be clinically unimportant because of the modest actual difference in medians. However, it is possible this modest difference actually represents a true difference in severity of illness between the two populations.

The presenting clinical complaint showed a slight decrease in penetrating trauma in the post-CPOE group. The reasons for the decrease in penetrating trauma are unknown.

The post-CPOE group demonstrated an increase in the number of patients discharged from the ED stabilization room to the computerized tomography and intensive care units, and a decrease in the number of patients admitted to a hospital floor bed. This might be evidence for the post-CPOE group being a more ill group.

We used patient disposition as a gross marker of neurologic function, assuming a discharge to home indicated independent function and a better neurologic outcome compared to those discharged to either nursing homes or rehabilitation facilities. This is an imprecise measurement of actual neurologic function, and actual measurement using neurologic outcome scales of each patient would have provided better definition of neurologic outcome. However, these data were not available in this retrospective study.

Although our primary endpoint, a difference in mortality, was not observed, our data do suggest that patients in the post-CPOE group had a decrease in functional outcome. Future studies will need to address functional outcomes as a primary endpoint.

Lastly, an unintended consequence of implementation of CPOE is significant changes in previously well established patient care workflow processes, especially for physicians. It is possible that a decrease in mortality rate might occur months to years after CPOE implementation as a consequence of physician adaptation and improvement in patient care workflow within the CPOE system. A follow-up study with the same methodology is warranted.

CONCLUSION

The implementation of CPOE in our hospital did not result in a change in mortality in ED critically ill or injured patients. There was a decrease in the proportion of patients discharged to home in the post-CPOE group, with a corresponding increase in the proportion of patients discharged to settings requiring higher levels of care.