Implementation of computerized physician order entry in National Guard Hospitals: assessment of critical success factors.

OBJECTIVE: The purpose of this study is to describe the needs, process and experience of implementing a computerized physician order entry (CPOE) system in a leading healthcare organization in Saudi Arabia.
MATERIALS AND METHODS: The National Guard Health Affairs (NGHA) deployed the CPOE in a pilot department, which was the intensive care unit (ICU) in order to assess its benefits and risks and to test the system. After the CPOE was implemented in the ICU area, a survey was sent to the ICU clinicians to assess their perception on the importance of 32 critical success factors (CSFs) that was acquired from the literature. The project team also had several meetings to gather lessons learned from the pilot project in order to utilize them for the expansion of the project to other NGHA clinics and hospitals.
RESULTS: The results of the survey indicated that the selected CSFs, even though they were developed with regard to international settings, are very much applicable for the pilot area. The top three CSFs rated by the survey respondents were: The "before go-live training", the adequate clinical resources during implementation, and the ordering time. After the assessment of the survey and the lessons learned from the pilot project, NGHA decided that the potential benefits of the CPOE are expected to be greater the risks expected. The project was then expanded to cover all NGHA clinics and hospitals in a phased approach. Currently, the project is in its final stages and expected to be completed by the end of 2011.
CONCLUSION: The role of CPOE systems is very important in hospitals in order to reduce medication errors and to improve the quality of care. In spite of their great benefits, many studies suggest that a high percentage of these projects fail. In order to increase the chances of success and due to the fact that CPOE is a clinical system, NGHA implemented the system first in a pilot area in order to test the system without putting patients at risk and to learn from mistakes before expanding the system to other areas. As a result of the pilot project, NGHA developed a list of CSFs to increase the likelihood of project success for the expansion of the system to other clinics and hospitals. The authors recommend a future study for the CPOE implementation to be done that covers the implementation in all the four NGHA hospitals. The results of the study can then be generalized to other hospitals in Saudi Arabia.

INTRODUCTION

Thousands of Americans die each year as a result of medical errors that could have been prevented, according to the Institute of Medicine report (IOM).[1] Beyond their cost in human lives, preventable errors also result in an estimated total cost of between $ 17 billion and $ 29 billion per year in the US hospitals. In response to the shocking IOM report, some healthcare organizations have introduced clinical information systems such as Computerized Patient Records (CPR) to improve outcomes, reduce medication errors, increase healthcare efficiency, and eliminate unnecessary costs.[2] Many hospitals have invested significantly to plan, procure, and implement these advanced systems, including the current focus on computerized physician order entry (CPOE).

CPOE represents an important step forward for healthcare organizations because it embodies a shift from traditional, paper-based care coordination activities to automation of the order entry processes. This shift can be an agent for change, eliminating confusing or illegible hand-written order documentation, minimizing transcription errors and reducing clinical mistakes.[3-10] However despite their knowledge, investments and best intentions, most health organizations have not realized a return on their investments.[11-13] In reality, less than 10% of the US hospitals have implemented CPOE according to a recent survey.[14] The deployment of health information systems including CPOE is especially challenging for Saudi hospitals because of high implementation cost, technical complexity, lack of information and communication technology (ICT) infrastructure, and lack of well-trained employees.[15]

In spite of the many advantages that information systems (IS) bring to organizations, many studies have found that IS project failures are very common. Only 32 per cent of IS projects succeed (delivered on time, on budget, with required features and functions) according to a Standish Group study.[16] It is estimated that around 44 per cent of IS projects partially fail with time and/or cost overruns and/or other problems. Around 24 per cent of IS projects are total failures and abandoned.

The failure rate of IS projects is even worse in the public sector reaching around 84 percent. The financial impact of the IS project failures is huge. Around $ 150 billion are wasted annually on IS failures in the United States and a similar number is reported in the European Union.[17] As a result, there is an expanding literature on IS project failures including both the theory and case studies. Some of the studies identified ‘Critical Success Factors’ (CSFs) that, upon careful consideration by the project team, increase the success rate of the IS projects.

CPOE system: Overview and benefits

CPOE is a process of electronic entry of physician's orders and instructions for the treatment of patients. These orders are usually communicated over the computerized patient record (CPR) system to other medical staff (nurses, therapists, or other physicians) or to the departments (pharmacy, laboratory, or radiology) responsible for fulfilling or documenting the order.

CPOE is not a technology, rather it is a workflow design of clinical processes that integrates technology to optimize physician ordering of medications, laboratory tests, and other clinical investigations.[18] CPOE uses clinical decision support systems and links to CPR systems to generate prompts and alerts during the ordering session to notify of potential errors such as contra-indicated medications or routes or duplicate orders.[4]

The functions of CPOE systems vary from one to another depending on the complexity of the system. For example, if we look at a basic CPOE system, it may simply offer a selection menu of drug names and doses or predefined order sets. Other applications may limit field entries for dosage control while others provide default values and templates that offer more guidance. Pull down menus may provide definitions, routes, or information about drug interactions. Some functions may be passive, requiring the physicians to search for a particular field; other functions are active and automatically provide needed data. More advanced applications integrate electronic medical records (EMRs) with surveillance systems that alert the physician of changes in patient vital signs and other clinical status issues.[4]

In spite of the benefits of CPOE, the agency for healthcare research and quality (AHRQ) identified 22 situations in which the CPOE system increased the probability of medications errors.[18] These situations fell into two categories: Information errors generated from fragmented information systems; and interface problems between humans and machines, where the computer's requirements are different from the way clinical work is organized. The AHRQ calls for careful and thoughtful implementation of CPOE systems to avoid facilitating errors.

In May 2001, thirteen CPOE experts from around the world met at a 2-day conference at a retreat called Menucha, for the purpose of identifying success factors for implementing CPOE.[19] A list of high-level considerations was generated to benefit organizations implementing CPOE which include: Motivation for Implementing CPOE, Costs, Integration of Workflow Processes, Value to Users, Vision, Leadership, Technical Considerations, Project management, training and support, and Learning/Evaluation/Improvement. The conference also listed some minor considerations for each of the high-level considerations.

About NGHA

The National Guard Health Affairs (NGHA) is a large health organization, which provides a modern medical care to all National Guard employees and their dependents. Under the umbrella of NGHA, there are four hospitals and sixty primary and secondary health centers around the Kingdom having 2000 in-patient beds in total. NGHA serves more than 2.5 million out-patients and around 60,000 in-patients annually. NGHA is organized in three regions: Central Region, Western Region and Eastern Region with the Eastern Region having two hospitals in Al Hasa and Dammam. NGHA is accredited by the Joint Commission International.

The purpose of this paper is to describe the needs, process and the experience of implementing a CPOE system in a leading healthcare organization in Saudi Arabia. This is very important due to the fact that there are very few articles addressing health information systems in Saudi hospitals in the literature.

METHODOLOGY

NGHA, as a result of an accreditation process, decided to give more emphasis on patient safety issues. Medication errors were an issue and a committee was formed to enhance patient safety. The committee reviewed research articles which in general support the adoption of CPOE solution to reduce medication errors. However, some research articles raised some concerns about the success of information systems implementations.[17] As a result, NGHA decided to go for a pilot project to assess the CPOE potential benefits and risks in a pilot department. The result of the assessment would determine whether CPOE is worth the investment and hence be rolled-out to other departments and hospitals or not. The project team focused on several aspects and considerations for selecting the Pilot unit including the following: The number of attending physician (small group managing most patients), the availability of physician champion or other advocates, how well-managed patient care unit, the unit includes highly specialized physicians, and the degree of IT literacy of care providers in the unit.

CPOE pilot project

The Adult intensive care unit (ICU) at King Abdulaziz Medical City in Riyadh was selected to be the pilot area as they met the selection creiteria mentioned above. The Adult ICU consisted of nine beds. On the provider side, there were approximately 25 physicians, 18 nurses, 3 respiratory therapists and there was a specialized pharmacy satellite located in the ICU area. The number of clinicians increased during the implementation due to a major expansion of the hospital.

Rather than investing in new “stand-alone” clinical information system, NGHA decided to use the CPOE features already in the CPR, which has been already used at KAMC across all in-patient and out-patient departments. Physicians are familiar with the system as they use it to view the results of the procedures and orders. Therefore, NGHA would benefit from the data integration between the CPR and the CPOE. This integration would provide decision support capabilities supported by the aggregate data within the CPR.

An ad hoc (Task-force) committee was formed to drive the implementation. This committee consists of multidisciplinary key stakeholders from various beneficiary departments: The Chairman of Adult ICU (project champion), CPR Team leader (project manager), and director of Pharmacy, the Nursing Manager, and Respiratory therapy manager. The committee assigned super users from ICU Physicians, Nurses and Pharmacists in addition to some application analysts from CPR department. The role and responsibility of the super user is to collect the data needed by ICU area in order to build and customize the order sets and protocols needed by ICU and to train the end user on how to use the CPOE. By the end of December 2006, the CPOE was implemented in the ICU.

Figure 1 shows the main features of the system, which include six screen shots from the CPOE system. One screen shows the chart review for a patient which includes access to all patient clinical data. The “Interactive Care Grid” screen include functions to display patient clinical data with clear indicators within a scalable period such as vital signs, active medications, and results and reports. The order list screen is used to approve the orders, design a clinical protocol, and review newly completed results. The Trends and Graphs screens are used to display patient clinical data with clear indicators using evidence based medicine.

Figure 1

Screen shots from the CPOE system implemented in NGHA

Assessing the CSFs in the pilot project

In order to ensure higher degree of success for the project, the project team addressed the following high-level (major) considerations developed by the 2001 Menucha conference which includes:[19] Motivation for Implementing CPOE, Costs, Integration of Workflow Processes, Value to Users, Vision, Leadership, Technical Considerations, Project management, training and support, and Learning/Evaluation/Improvement. Table 1 shows high-level considerations along with the actions taken by NGHA to address the considerations.

Table 1

Major CSFs[19] and the actions taken by NGHA to address the critical factors

After the finalization of the pilot project, a survey was conducted to check if the clinicians agree with list of minor critical success factors developed by the Menucha conference.[19] These factors fall under the high-level considrations listed in the same study. The demographic data of the respondents is shown in Table 2.

Table 2

Frequency distribution of sociodemographic variables (n=43)

Purposive sampling was used to choose the physician, nurses, and pharmacists in the ICU. A two-part structured questionnaire was developed. Part one included some questions about the demographic information including respondent age, gender, position, working area, and other question. Part two included thirty two factors to measure the precieved importance of the success factors for CPOE implementation. The reliability of the questionnaire was measured using the coefficient alpha; it was 95.2%. Sixty survey forms were distributed based on the physician, nurses, and pharmacists number in the ICU. A cover letter explaining how to respond to the questionnaire items was attached, of which 43 were returned (71.6%) and all of them were valid. A five-point Likert scale (1 = extremely not critical, 2 = not critical, 3 = neutral, 4 = critical, 5 = extremely critical) has been used in this research. The interpretation will be as shown in Table 3.

Table 3

The interpretation of the results based on a five-point Likert scale

Expanding CPOE implementation to other NGHA departments and hospitals

Phase 2 of the CPOE project did not start until 2008, which is three years behind schedule due to some reasons including: The CPR system was acquired by other companies during this period which resulted in different management team and different commitments, the system was faced by some physician resistance which kept the team busy addressing the new raised issues, and the initial team size was small and it was found to be risky to start the rollout process.

After addressing the above issues, a new project structure for the CPOE was approved as shown in Figure 2. Due to the highr resistance faced by some physicians from the CR hospital, the project team adjusted the project plan so that the Al Hasa hospitals and Dammam hospitals will go-live before other hospitals. This decision was also supported by the smaller size, smaller number of physicians, smaller number of specialities and the enormous support practiced by the medical director. The CPOE system was implemented successfully with a “big-bang” approach in the Eastern Region hospitals (Al-Hasa, and Dammam) in September 2009. The success of the CPOE in these two hospitals created a pressure on the Riyadh hospital to resume the its implementation and as a result the physicians’ resisitance reduced.

Figure 2

CPOE project-management structure

Meanwhile, the diversity of specilities and the size of Riyadh and Jeddah hospitals mandated a staged implentation approach for the CPOE as shown in Table 4. It is expected that by the end of 2011, CPOE will be implemented in all NGHA departments and hospitals.

Table 4

The phasing of CPOE implementation in Riyadh and Jeddah hospitals

RESULTS

Results of the ICU pilot project

The result of the ICU survey for measuring the perceived degree of criticality (importance) of the Menucha CSFs for CPOE at KAMC, is shown in Table 5. The overall degree of criticality assessment for the CSFs in the implementation of CPOE Systems falls in the range of “Critical” where the main factors means average = 3.64 and standard deviations average = ± 0.90; the value of the standard deviation shows acceptable deviation of respondents regarding their overall assessments, the thirty two factors were sorted based on their means which shows that factors Training (Before Go live), Adequate clinical resources and ordering time have the highest degree of criticality according to their means (4.0 , 3.9, and 3.88 and standard deviations of ±1.046 , ±0.75, and ±0.98, respectively). However, the the existence of verbal order after implementation and the ability to access the sytem remotely were showing the lowest degree of criticality among the factors at means of 3.35, 2.81 and standard deviations of ± 1.09 and ± 1.06, respectively.

Table 5

The degree of perceived importance of CSFs by ICU staff for the CPOE implementation

The results showed that the ICU staff agreed with most of the conisderations developed by the 2001 Menucha conference. This is encuraging to show that these considerations could be used throught the project for expanding the CPOE implementation to other NGHA clinics and hospitals.

During the pilot project, NGHA was confronted with a lack of expertise internally in the area of health IT in general and the CPOE systems specifically. NGHA foucsed on developing CPOE core team and have provided the team with the necessary training during the period 2006-2009. During that time the level of physicians’ resistance was at its peak. The major complain was related to the fact that switching from handwriting to CPOE would require a significant change in the physician's work flow. Moreover, in the early adoption of CPOE, the system required physicians to spend more time placing orders on the computers.

Lessons learned

CPOE is not a pure IT project rather it is a clinical IT project and as a result it should be led by physicians and not IT staff. The benefits of CPOE systems will be realized if the system is well integrated with the hospital-wide CPR.

It is very important when introducing CPOE systems in hospitals to start the implementation in a pilot area. This will enable the organization to test the system without putting patients at risk.

Hospitals adopting CPOE ssytems should expect physicians’ resistance. In order to prevent this from failing the project, strong leadership support is needed. Organizations should be ready for a work flow change. The project team should focus on quick wins as this would reduce the level of resistance.

Training should be done properly and the need, time and resources for training should not be underestimated. As the number of physicians to be trained is larger than the implementation team, “train-the-trainer” approach should be used. Hospitals should mandate physicians to attend the required training.

Hospitals should control scope changes. Some end users will keep requesting additional features that are not planned for early phases. A process for scope changes should be established.

The “Go-Live” day is not the end of the CPOE project. Many issues will come months later as end users gain more knowledge about the system and its capabilities.

Hospitals undertaking CPOE implemntation should have address some of the CSFs prior to the implementations.

Limitations of the study

The authors realize the small sample size used in the survey. However, the survey was important for NGHA to assess the project in order to make a descision weather it should halt the project or expand it to other NGHA clinics and hospitals. However, as the literature lacks similar studies done in Saudi Arabia, it is important to share our findings to benefit other hospitals undertaking similar projects.

The study could be generalized if another survey after the CPOE is fully deployed in all NGHA departments and hospitals in order to safely generalize the study.

Validation of the final CSFs by correlating them to providers’ satsfaction or success of implementation is essential in CSF selection. This was not done in this study and should be planned for future studies.

DISCUSSION AND CONCLUSION

To ensure a higher degree of success for CPOE implementation, organizations should consider certain critical success factors. CPOE projects should not be viewed as another ICT project rather should be viewed as a business project. It is important to implement the CPOE in a pilot department in order to test the system without putting patients at risk and learn from mistakes. Lessons learned would then be used in the roll-out process. Based on the analysis conducted in the previous section, it is clear that healthcare environment's mission criticality and unique characteristics have influenced the factors degree of criticality, which was clearly identified by the sampled staff responses.

Training is a major success factor for CPOE projects. Organizations should not underestimate the need, time, and resources for training. This should include functional training as well as technical training. Organizations should consider using the “train the trainer” approach, where training was given to selected users (key users) who, in turn, conducted training sessions to other users.

The importance of the time for ordering has been stressed by the respondents. Time is playing a significant role especially when the physician is entering the orders while he or she is attending with the patients. Appropriate setting and configuration of the orders sets with minimum number of screens displayed for the user ease and facilitate the process of order entering and as a result it will reduce the time of the order entry. The project team made a balance between customization and standardization where physicians of a similar specialty would have a common CPOE screen, which might be different of other specialties.

Hospitals should also make sure that they have the right technical infrastructure prior to considering CPOE implementation. In NGHA case, project team worked hard to ensure the technical infrastructure before the “go-live” of the CPOE. The technical infrastructure included network wireless access points and laptop workstations.

The integration of CPOE and CPR is very important to the success of the CPOE deployment. Such integration provides decision support capabilities supported by the aggregate data within the CPR eliminating the need for integration engines to link CPOE with other modules like pharmacy, lab, radiology, and nursing.

Hospitals should adopt strong project management techniques and tools to ensure the success of CPOE deployment. Organizations should control scope changes. Inevitably, employees will keep requesting additional features that are not planned for the first phase. Organizations should establish a process for scope changes and decisions on making the changes should be based on a cost/benefit analysis. The “Go-Live” day is not the end of the CPOE project. It is essential that the vision of the CPOE project be clear to the project team and to the concerned departments.

Finally, the CPOE system is not limited to physician order entry; the future holds promising features to physicians such as automating their progress notes, history and physical and other features to ensure a complete electronic patient record. NGHA has plans to bring these new features after the completion and stabilization of the CPOE.