Effectiveness of Evidence-based Pneumonia CPOE Order Sets Measured by Health Outcomes.

OBJECTIVE: Evidence-based sets of medical orders for the treatment of patients with common conditions have the potential to induce greater efficiency and convenience across the system, along with more consistent health outcomes. Despite ongoing utilization of order sets, quantitative evidence of their effectiveness is lacking. In this study, conducted at Advocate Health Care in Illinois, we quantitatively analyzed the benefits of community acquired pneumonia order sets as measured by mortality, readmission, and length of stay (LOS) outcomes.
METHODS: In this study, we examined five years (2007-2011) of computerized physician order entry (CPOE) data from two city and two suburban community care hospitals. Mortality and readmissions benefits were analyzed by comparing "order set" and "no order set" groups of adult patients using logistic regression, Pearson's chi-squared, and Fisher's exact methods. LOS was calculated by applying one-way ANOVA and the Mann-Whitney U test, supplemented by analysis of comorbidity via the Charlson Comorbidity Index.
RESULTS: The results indicate that patient treatment orders placed via electronic sets were effective in reducing mortality [OR=1.787; 95% CF 1.170-2.730; P=.061], readmissions [OR=1.362; 95% CF 1.015-1.827; P=.039], and LOS [F (1,5087)=6.885, P=.009, 4.79 days (no order set group) vs. 4.32 days (order set group)].
CONCLUSION: Evidence-based ordering practices have the potential to improve pneumonia outcomes through reduction of mortality, hospital readmissions, and cost of care. However, the practice must be part of a larger strategic effort to reduce variability in patient care processes. Further experimental and/or observational studies are required to reduce the barriers to retrospective patient care analyses.

Introduction

Efforts to employ evidence-based medicine to treat well-researched patient conditions, heighten core clinical measures compliance, and improve quality through process standardization have been ongoing for decades. Studies have demonstrated successful reductions in the rate of adverse drug effects, which have encouraged further advances in such healthcare information technologies as the automated venous thromboembolism (VTE) alerts developed at Brigham and Women’s Hospital in Boston [1]. Indeed, one of evidence-based medicine’s greatest successes has been medication order sets available to physicians via computerized physician order entry (CPOE) applications. Approved by multi-disciplinary professional committees typically consisting of physicians, nurses, and pharmacists, these CPOE applications have enabled treatment standardization for certain patient conditions, i.e. pneumonia, congestive heart failure, acute myocardial infarction, via pre-defined templates. These templates are found within CPOE patient view menus among order options and, depending on specific application, may appear in a searchable and/or categorized drop-down list, with only a few clicks necessary to pull up a set and place an order. Order sets typically include medication orders and care giver communications such as nursing and dietitian instructions. Sets represent a complete care document that includes comprehensive evidence-based patient care orders for a given medical condition.

CPOE instructions are easy to access, decrease delays, reduce errors, and improve inventory control. This resonates in the national debate over slowing the growth of healthcare expenditures through more focused applications of information technology not only in CPOE, but also in electronic medical record (EMR) and clinical decision support systems (CDSS) to demonstrate meaningful use, as mandated in the 2010 U.S. Patient Protection and Affordable Care Act. Aided by sophisticated information technology, expectations of CPOE benefits have moved beyond efficiency and are now linked to sharpening diagnoses, improving clinical outcomes, and tracking treatment plans through the continuum of patient care. Despite differing and evolving expectations of CPOE, and its order set component specifically, standardized ordering practices are widely deployed throughout the healthcare sector. Yet, few longitudinal clinical studies have empirically explored and validated order set utilization to treat eligible patients, from the healthcare cost and patient outcomes perspectives.

Early limited scope studies in treating pneumonia found that order sets were effective in reducing mortality, length of hospital stay, and core measures compliance, without affecting readmissions [2,3]. However, none of these studies analyzed large data sets spanning several years of patient encounters. Larger data set analysis of pneumonia patients is needed, as this disease is common in older adults and will increase in magnitude as baby-boomers grow older. The analysis should include tracking the impact of health informatics on a sizable patient population. Increasing size of the data set in a study will make results appear more convincing for public health informaticists concerned with the overall impact of biomedical informatics on patient populations’ health outcomes.

In this study, we examine the effectiveness of pneumonia order sets in a major community integrated healthcare delivery network using patient history from four Advocate Health Care hospitals for the 2007–2011 period. Conducted at Advocate Health Care, one of our goals was to explore quantitatively the effectiveness of pneumonia order sets, as well as what successful implementation of an order set means in the context of a larger process of patient care practices standardization. Based in Oak Brook, Illinois, Advocate is the largest integrated delivery network in Illinois (USA), with 12 hospitals, 3,500+ beds, and over 250 sites of care.

Background and Significance

Medication allergy checking, dose calculations, and drug interactions are some of the most common physician actions performed when ordering medications. These activities are also some of the most error-prone treatment stages, especially in fast-paced community hospital environments [4]. As Ahmad et al. [5] and Payne et al. [6] showed, CPOE success is measured by (1) the percentage of orders entered directly into CPOE by providers and (2) the overall utilization of the order sets. Cowden et al. [7] conducted an order set quality improvement study with the goal of combining two orders and evaluating a combined chi-squared measure to predict order set correspondence to an ordering pattern. Performed at Ohio State University Medical Center, the authors concluded that a large percentage of medication orders qualified for an order set, meaning that evidence-based guidelines were available to consider standardization of ordering practices.

Dixon and Zafar [8] defined the theoretical foundation of order sets and their potential effectiveness in standardized treatment in a study sponsored by the U.S. Department of Health and Human Services. Ballard et al. [9] conducted a congestive heart failure order set study at Baylor Health Care System and concluded that evidence-based prescription methods decrease mortality, length of hospital stay, and rate of readmissions. Best et al. [10] found reduced interval rates in initiation of antibiotic therapy from the time of diagnosis in treating febrile neutropenia, with a potential for greater success by swifter clinical response. Chisolm et al. [11] studied 790 pediatric asthma patients at Columbus Children’s Hospital and concluded that those who received medications via order sets were more likely to receive spinal cord stimulators (SCS) and pulse oximetry (Pulse Ox) than patients in the control group, resulting in better health outcomes. In a controlled trial of 179 diabetes mellitus and inpatient hyperglycemia patients at an academic hospital, the primary mean percentage of the glucose readings per patient was reduced to 60-180 mg/dL [12]. The result of this experimental study, based on a custom medication order (control group) and a CPOE admission order set (intervention group), indicated a positive outcome from use of the standard medication order sets.

Wright et al. [13] analyzed patterns in order set utilization among several leading healthcare facilities in the Northeast United States. They found that order set governance and maintenance were costly, and only a handful of available order sets incurred high utilization. Even so, the primary reasons for order set use included patient safety and ordering efficiency. Through 71 physician interviews, another study found cost concerns to be the top factor in predicting order set utilization [14]. Yet physician engagement remains a critical success factor in order set implementation. Adventist Health System in Florida attributed physician engagement as the most important reason for the success of a 2.5-year project to address 80% of the diagnosis-related cases with an order set approach [15]. To address the cost and complexity of order set selection and governance, in 2007 the Mayo Clinic instituted a comprehensive order set review process that included formal committee evaluation, approval process, and mid-term progress checks on utilization and patient outcomes [16]. The Mayo Clinic discovered that teamwork and inclusive participation in the formal review process by clinicians increased buy-in, order set utilization, and subsequently had positive impact on patient safety. Bekmezian et al. [17] quantitatively measured the perceived benefits of pediatric admission order sets (PAOS) among 97 medical residents at the University of Southern California and Children’s Hospital Los Angeles. Eighty-nine percent of residents approved PAOS; 58% admitted using it all the time. Eighty-eight percent of residents reported that PAOS saved time, 93% believed it to be convenient, and most reported less need for communication and clarification with nurses and secretaries.

Other than health outcomes, the best measures of treatment and process effectiveness are core compliance data, which serve as proof of the order set implementation’s positive impact in clinical settings. Broussard et al. [18] conducted a pediatric sedation drug dosing study at Louisiana State University Health Sciences Center in Shreveport using paper order sets to compare 26 patient intervention cases to the control group of 42 sedations. Fully documented compliance cases increased from 32% to 69% and several medication dosages fell within the recommended range. Ballard et al. [2] conducted an observational study of pneumonia patients to examine hospital order set use by discharge month, severity of illness, and risk of mortality. Among 3,301 patient cases, order set use increased by 55% and significantly improved in-hospital mortality [95% CF: 0.66 (0.45 - 0.97)] and core measures compliance [95% CF: 1.24 (1.04 - 1.48)], without affecting the 30-day readmission rate.

In a study of 120 patients with a diagnosis of septic shock determined within the emergency department of a 1,200-bed academic medical center [19], patients in the experimental CPOE group were more likely to: (1) receive intravenous fluids, (2) receive fluids of >20 mL/kg body weight before vasopressor administration, and (3) be treated with an appropriate initial antimicrobial regimen, compared to patients in the control group. Thiel et al. [20] performed a severe sepsis study at the 1,200-bed academic Jewish-Barnes Hospital. Bacteremic patients in the experimental CPOE group received more intravenous fluids in the first 12 hours after onset of the hypotension and were more likely to receive an appropriate dose of antibiotic therapy, compared to patients who received orders via manual practices outside of CPOE. Mortality statistically decreased in the experimental CPOE group, along with the hospital stay.

Fleming et al. [3] measured utilization and health outcomes for 4,454 patients after implementation of a community-acquired pneumonia order set at the Baylor Health Care System. Analysis showed significant reduction in length of hospital stay, 30-day mortality, and direct cost, with a 75% increase in compliance. O’Connor et al. [21] conducted a deep vein thrombosis (DVT) prophylaxis quality improvement study at the 750-bed community hospital in Mississauga, Ontario. Paper medication order sets were voluntarily used by internists without prior education on the benefits of the evidence-based approaches to medicine. Order sets were used to prescribe admission medications to 10.9% of the patients, who were 23.4% more likely to receive DVT prophylaxis than patients in the control group.

Overall, the literature supports CPOE order set effectiveness in increasing core measures compliance, decreasing mortality and morbidity, and shortening the length of hospital stay. Empirical evidence demonstrates beneficial application of evidence-based medication prescribing practices through utilization of electronic CPOE-based order sets, as long as appropriate patient groups are targeted and the theoretical basis for selection of the order sets is justified by evidence and approved by interdisciplinary teams of clinicians involved in patient care. However, the number of studies that examined pneumonia order sets is more limited and involved smaller patient populations, leaving room for exploring the subject of order set effectiveness in a large community-based healthcare system.

Methods

Data for the study was obtained via SQL queries from the Enterprise Data Warehouse (EDW) that receives information stored in EMRs, patient registration, clinical and financial decision support, and other patient care and data analytics applications. The EDW uses Kimball architecture and runs on SQL Server database. Pneumonia order sets researched in this study are available to all physicians with access to CPOE, but are not published externally. Only electronic sets available within hospital CPOE were included in this study. Paper sets were excluded. While paper sets could have provided additional insight into health outcomes as results of evidence-based standardization of patient care practices, such an element would have introduced manual work to a study based on analysis of aggregate historical data, and some of the necessary fields stored in the EDW would not have been available for patient encounters where paper sets were applied.

In this causal comparative study, we analyzed pneumonia patient data for the 2007–2011 period. Our goal was to determine order set effectiveness as applied to “order set” and “no order set” groups of patients based on health outcomes, 30-day readmissions, length of hospital stay, and supported by comorbidity analysis. We focused on patients with community-acquired pneumonia based on large patient volumes, higher utilization of order sets than other conditions, and inclusion in the current key result areas of quality metrics at the research site. As part of a comparative study of patient history, the order set group represented patient encounters where providers placed pneumonia orders using CPOE sets, while the no order set group represented all other pneumonia treatment orders where physicians chose custom ordering methods and did not employ sets.

Mortality was confirmed by selecting discharge codes complying with ICD-9-CM diagnosis codes listed under AHRQ quality indicators (IQI #20) pneumonia mortality definitions. Eligible “expired” codes were converted to a binary value of 1, with other codes converted to 0. Readmission was a “yes/no” binary field, while length of stay (LOS) was a calculated field between the date/time values of admission through discharge. We used the total Charlson Comorbidity Index (CCI) to represent comorbidity. In this study, CCI could play a dual role: (1) it could help us to explain LOS as an index of pre-existing conditions, thus introducing comorbidity adjustment, or (2) it could serve as a measure of treatment complications. Due to the lack of a clear definition of comorbidity as a variable in this study, the results are only applicable to discussion. Comorbidity calculations are separate due to the lack of a reliable measure to adjust results by comorbidity, with CCI serving in the dual pre/post-treatment complication roles to help explain potential reasons for mortality and LOS changes.

The main independent (cause) variable in this study was utilization of the two order set groups. Health outcomes – mortality, readmissions, LOS, and comorbidities - were defined as dependent (effect) variables. Race, age, and sex were mediating variables analyzed in conjunction with order set utilization to determine combined significance in predicting health outcomes. De-identified patient encounters were obtained via queries against the enterprise data warehouse containing records from CPOE and patient accounting applications. The data were entered into and manipulated in Excel to adhere to the following inclusion and exclusion criteria:

Adults over the age of 18;

Patients with primary or secondary diagnosis of community-acquired pneumonia; and

Exclusion of psychiatric and obstetrics patients.

Due to the retrospective nature of the study and limited harm to patients, IRB approval was obtained under the Expedited de-identified category, with a HIPAA waiver form requiring no consent from patients. The data obtained from the EDW contained no identifying information, such as social security numbers or names. Records were identified via medical record number (MRN), which is internal to EMR/CPOE applications.

Pneumonia patients were assigned to the order set and no order set groups based on their diagnosis and physicians’ ordering preferences. Data were subsequently loaded into IBM SPSS statistical software for analysis. Binary logistic regression with a chi-squared option was employed to measure mortality and readmission outcomes. The Mann-Whitney U test was utilized to test the null hypothesis for LOS, while a one-way ANOVA was employed to compare the mean LOS scores. The latter two tests were also used to measure comorbidity. Fisher’s exact test was employed as a secondary method of measuring statistical significance, due to the small order set group size.

In the mortality study, population N of the “order set” group was 362, while N of the “no order set” group was 4,725. In the 30-day readmissions study, population N of the “order set” group was 556, while population of the “no order set” group was 4,531. In the LOS study, population N of the “order set” group was 362, and 4,725 in the “no order set” group. In the comorbidity/complications study, the numbers were 556 and 4,427 between the two respective groups of patients. The differences in sample group sizes between studies can be explained by the availability of patient records on readmissions and CCI data in the enterprise data warehouse.

Low utilization of the order sets in hospitals (around 4% of all pneumonia medication orders) is a challenge faced in all CPOE studies. This limits the categories in which statistically significant results can be reported. In this study, the outcomes for all participating hospitals needed to be combined, which limited analysis of the order set content variances among hospitals, although such variances were small. Similarly, due to the retrospective nature of the study, as well as the lack of a research-oriented CPOE design, limited observations regarding causal relationships among clinical records associated with each patient encounter could be made. In practice, order set use is optional, contributing to the low percentage of orders placed via this route and the disproportionally large size of our no order set group, where custom ordering methods have been typically employed. This indicates the co-existence of multiple pneumonia ordering practices. The choice of method to place pneumonia orders is entirely in the hands of healthcare providers. Despite using a combination of different pneumonia data sets, all of our data were evidence based and approved by clinicians to be available in CPOE for the purposes of treating pneumonia. Table 1 is an example of the actual content of a pneumonia order set utilized at one of the participating hospitals. Entries are grouped and appear in the same order as they do in CPOE, but comments regarding order placement logic (i.e., references to allergy-related medication choices, nursing home patients, smoking status, etc.) have been removed for brevity, leaving the primary focus on the content of a typical pneumonia order set.

Table 1

Pneumonia order set example

Component	Order Details
Admission
Admit Order	Inpatient Admission
Admit Order	Place in Observation Services
Activity
Activity Patient	Complete Bed Rest
Activity Patient	Up ad Lib
Activity Patient	Up ad Lib, Assistance Needed
Diet
General Diet
Diabetic 1,800 Calorie Diet
Cardiac Diet
NPO
Medication
Ceftriaxone (ceftriaxone (Rocephin))	1 gm, IVPB, Q24H, Give first dose stat
Azithromycin (azithromycin (Zithromax))	500 mg, IVPB, Daily, Give first dose stat
Doxycycline (doxycycline (Vibramycin))	100 mg, IVPB, Q12H, Give first dose stat
Moxifloxacin	400 mg, IVPB, Infusion, Q24H, Give first dose stat
Vancomycin	1,000 mg, IVPB, Stat
Vancomycin	Pharmacist to dose based on weight and renal function
Piperacillin – Tazobactam (Zosyn) Infusion Med Set
Piperacillin – Tazobactam (Zosyn)	3.375 mg, IVPB, Q8H
Acetaminophen	325 mg
Acetaminophen	500 mg
Acetaminophen	650 mg
Acetaminophen	1,000 mg
IV Fluids
Saline Lock Insertion	Routine saline lock care
Lactated Ringer’s	1,000 mL, IV, IV Soln
Sodium Chloride 0.9%	1,000 mL, IV, IV Soln
Dextrose 5% - 0.45% NaCl	1,000 mL, IV, IV Soln
Respiratory
Blood Gas	Routine, Arterial Blood
Pulse Ox Spot Check	Once
Pulse Ox Spot Check	Daily, if patient is on oxygen
Oxygen
Labs
Blood Culture (BLC)	Stat Draw (Draw Stat/Perform Routine)
Blood Culture (BLC)	Stat Draw (Draw Stat/Perform Routine)
Gram Smear [GRAS]	Next Draw/Specimen Collection, Lung, Sputum
Respiratory Culture/Smear (RTCS)	Next Draw/Specimen Collection, Lung, Sputum
Legionella AG Urine (LEGEIA)	Next Draw/Specimen Collection
Influenza Rapid AG (FLUAG)	Next Draw/Specimen Collection, Nasopharyngeal Washing
CBC with Automated Differential (CBCA)	Next Draw/Specimen Collection
Comprehensive Metabolic Panel (CPNL)	Next Draw/Specimen Collection
Strep Pneumonia Antigen	Next Draw/Specimen Collection
Procalcitonin	Next Draw/Specimen Collection
Bronchial Alveolar Lavage (Mini Bal)	Routine
Radiology
XR Chest PA, Lateral 2V	Stat, Transport Mode, Portable, Reason for Exam: Pneumonia
Nursing
Vital Signs Per Unit Routine
Nursing Communication Order	Initiate smoking cessation education
Nursing Communication Order	Oral care protocol
Nursing Communication Order	Elevate head of bed 35 degrees
Consults
Discharge Planning Evaluation Adult

Results

Mortality

One of our goals was to determine, quantitatively, whether utilization of pneumonia order sets helps lower inpatient mortality. All tests were conducted to determine whether the null hypothesis could be rejected. The binary logistic regression method revealed that 6.6% of patients in the order set group (N = 362) died versus 11.3% in the no order set group (N = 4,725), OR = 1.787, 95% CF 1.170 – 2.730, χ2 = 7.402 (P = .061). The results approached statistical significance. Due to the relatively small size of the order set group, a more accurate and possibly more appropriate two-tailed Fisher’s exact test was used as an alternative to Pearson’s chi-squared method, with a statistically significant outcome of p = 0.05. Patients in the order set group, whose medications were ordered via pre-defined sets, were nearly twice as likely to survive compared to patients in the no order set group; thus, the null hypothesis for pneumonia mortality was rejected. Mortality outcomes are summarized in Table 2.

Table 2

Pneumonia mortality as a total for all patients in the study

Outcomes and Measures	No Order Set Group	Order Set Group	Pearson Chi-squared (χ2)	2-sided Fisher’s Exact	1-sided Fisher’s Exact	EXP (β) – Binary Logistic Regression
Mortality = yes	532	24
Mortality = no	4,193	338
Percent (Mortality=yes) / total	11.3%	6.6%	7.402 (P =.061	0.050	0.034	1.787 [1.170 and 2.730]

Given the limitation of fully tracking pneumonia patient encounters from admission to discharge on an aggregate basis, it is likely that other factors may have induced mortality among these patients, so utilization of order sets is likely not the only variable impacting the differences between our two groups. Comorbidity could serve as one such variable showing that patients in the order set group were “healthier” than those in the no order set group. The results of the comorbidity study are given in the Comorbidities/Complications section below.

Readmissions

Another study goal was to determine whether placing pneumonia orders via sets helped reduce the rate of 30-day hospital readmissions. As in the mortality examination, statistical manipulations were performed to test the null hypothesis. Only 10.8% of patients in the order set group (N = 556) were readmitted within 30 days, compared to 14.7% of patients in the no order set group (N = 4,531). The odds ratio EXP(β) was 1.362 at the 95% confidence interval [1.015 – 1.827, χ2 = 4.274], at P < .05. The results were statistically significant. The group sizes necessitated verification of the Pearson chi-squared test by determining the value of Fisher’s Exact test, which at 0.041 was not significantly different from the Pearson test. The differences in sample group sizes between studies can be explained by the availability of patient records on readmissions and CCI data in the enterprise data warehouse (EDW). Patients in the no order set group had an approximately one-third higher probability of being readmitted, compared to patients in the order set group; thus, the null hypothesis was rejected. Readmission outcomes are summarized in Table 3.

Table 3

30-day readmissions with pneumonia

Outcomes and Measures	No Order Set	Order Set	Pearson χ2	2-sided Fisher’s Exact	1-sided Fisher’s Exact	EXP (β) – Binary Logistic Regression
Readmission = yes	579	54
Readmission = no	3,952	502
Percent (Readmission =yes) / total	14.7%	10.8%	4.274 (P =.039)	0.041	0.020	1.362 [1.015 and 1.827]

Length of Stay

One-way ANOVA and a Mann-Whitney U test were employed to address whether application of order sets in the clinical settings can help reduce the LOS. The mean LOS for the order set group (N = 362) was 4.32 days versus 4.79 days for the no order set group (N = 4,725), indicating that pneumonia patients benefitted from utilization of the order sets by spending roughly 0.5 days less in the hospital. The one-way ANOVA test was significant, F(1,5087) = 6.885, P = .009. The Mann-Whitney U test confirmed rejection of the null hypothesis: [U(1,N = 5,087) = 1,148,309; P = .001], indicating the benefit of a shorter stay among the order set group.

In some cases, expired patients may have contributed to shorter LOS outcomes in this study. In order to adjust for mortality, the same calculations were repeated with expired patient data excluded. The results remained consistent, showing shorter LOS for the order set group (N = 338) versus the no order set group (N = 4,193): mean 4.26 (order set group) versus 4.71 (no order set group). The one-way ANOVA test was significant, F(1,4531) = 6.545, P = .011. The Mann-Whitney U test confirmed rejection of the null hypothesis: [U(1,N = 4,531) = 1,010,258; P = .001].

Comorbidities/Complications

The mean CCI score among the order set group (N = 556) was 2.13 versus 2.40 for the no order set group (N = 4,427). CCI is a total score attached to each patient encounter (where available) that was computed by adding individual weights of complications found in a patient. This score does not indicate whether a patient arrived with some of these conditions or acquired them during the hospital stay. Therefore, this score has a dual meaning: patients in the order set group were either healthier, if CCI represented pre-existing conditions, or had fewer complications following hospital discharge. The one-way ANOVA results were significant: F(1,4983) = 5.954, P = .015. The Mann-Whitney U test confirmed rejection of the null hypothesis: U(1,N = 4,983) = 1,153,767.5; P = .014.

Discussion

A retrospective study does not allow for variable control, selection of the order set content, or other attributes of an experimental study. However, such data offered an exceptional opportunity for the analysis of a large volume of patient encounters over time. These studies may identify certain trends in outcomes, something that many organizations cannot track based on information currently stored in their CPOE applications and/or clinical data warehouses. Our mortality and readmission outcomes showed that, in our patient sample, the risk of death and the possibility of hospital readmission for the same or related symptoms almost doubled over the sample period due, in part, to the lack of consistency in prescribing standard orders and medications – among other possible factors not examined in this study. The LOS for pneumonia patients was 0.5 days shorter for the order set group, indicating a potential for lowering healthcare costs in the era of value-based reimbursement and accountable care organizations (ACO), as well as higher patient and family satisfaction with being discharged earlier. This does not even take into account other positive outcomes resulting from shorter LOS, such as a shorter exposure to all the possible pathogens within a hospital environment.

Lower comorbidity among the order set group may indicate that patients in the no order set group were sicker if the CCI was viewed as a total reflection of pre-existing health conditions. However, a lower CCI may also indicate fewer complications as the result of advance pneumonia treatment orders and medication prescription standardization. Currently, no better method exists by which to assess the effects of evidence-based CPOE ordering practices on complication outcomes, so this comorbidity analysis study may be viewed as a reference study conducted as an attempt to help support and explain mortality, readmissions, and LOS health outcomes.

Statistics-based studies of large numbers of patient encounters do not always account for numerous other facets of patient care that influence outcomes. Most versions of CPOE and clinical data warehousing applications do not allow researchers to track patient records in their entirety, while tracking all treatments and variables involved with each patient. Thus, it is reasonable to assume that order set utilization was not the only factor that influenced positive or negative outcomes of mortality, readmissions, and LOS in this study. The study also did not include variables for undesired/unexpected side effects from care standardization practices. Due to the low utilization of order sets across the inpatient care facilities in this study, all data were combined; we disregarded the minor differences in the order set content between facilities, as they did not appear to influence the results significantly, as evidenced from data analysis pertaining to individual hospitals participating in this study. Despite the differences, all order sets were required to be evidence-based and signed off departmentally, following formal review by clinicians. Combination of the above factors could have had unknown effects on the validity of the outcomes, yet the findings indicate a possible trend in the positive effects of evidence-based medicine on pneumonia treatment.

Utilization of order sets in the treatment process for pneumonia is only one of many factors in the overarching variability reduction process aimed at improving the quality of patient care through adherence to evidence-based medical practices. Variability reduction may include initiatives such as shared governance among clinicians to target specific quality improvement efforts, selection of the most effective evidence-based medicine applicable to local clinical settings and culture, making decisions on selection and maintenance of the order sets, etc. The general availability of an order set, by itself does not mean application of evidence-based approaches to treating patients. Indeed, lack of governance around order sets may cause low buy-in, lower core measures compliance, and potentially dangerous side effects from improper bundling of medications and other orders.

Community-acquired pneumonia is a significant public health concern. The Centers for Disease Control and Prevention (CDC) reports that there were 49,597 pneumonia-related deaths in 2010 (the most recent publicly available data set), making the combination of pneumonia and influenza the ninth largest cause of mortality in the United States [22]. The majority of these cases affected people age 65 years and older. There were 1.1 million inpatient pneumonia discharges in the United States in 2010, with an average LOS of 5.2 days [23]. The reduction of approximately 0.5 days in LOS attributed to order sets in our study represents about 10% of the national average LOS for pneumonia patients, as listed in CDC pneumonia statistics [23]. In the era of patient-centered care and value-based reimbursement, a 10% reduction in LOS means a significant reduction in associated healthcare costs, as well as higher patient satisfaction. Our odds ratio finding of 1.362 for hospital readmissions with the diagnosis of community-acquired pneumonia indicates approximately 30% fewer patients leaving hospitals without full recovery and the potential for spreading the disease. In this study with a total of 633 readmitted patients over a five-year period, 30% readmission prevention rate translates into 229 fewer patients carrying a risk of pneumonia transmission into the community. Given the total number of pneumonia cases in the U.S., the potential for increased efficacy from order set implementation of evidence-based treatment guidelines is even greater on a more global scale. The implication of our findings may be of even greater importance to the most affected patient population suffering from pneumonia, namely 33,700 nursing home residents, which represented 2.3% of the United States nursing home population in 2010 [23]. Thus, our study outcomes may be beneficial to healthcare practitioners and scholars across many disciplines and patient populations.

Limitations

As with any retrospective study based on historical data, we did not have an opportunity to control all variables leading to pneumonia health outcomes. The reader should assume that other clinical factors beyond the scope and coverage of this study played a role in determining mortality, readmissions, and length of stay. However, outcomes of this study provide indication that evidence-based guidelines for treatment implemented via standardized ordering practices have a positive impact on outcomes. Order set design assumes physicians’ ability to select and/or deselect elements within order sets, and there is no reliable way of tracking all changes on an aggregate basis without manual review of each patient encounter, so there may have been certain differences between orders placed for patients who participated in this study. The dual role of complications and CCI, covered in the Discussion section, indicates limited effect of analyzing comorbidity, because it could mean that patients were either healthier/sicker prior to admission or came out healthier/sicker after being treated. Lastly, greater access to more granular data could help further categorize patients, their health conditions, and what unit(s) of the hospital in which they received treatment.

Conclusion

Despite wide availability and sporadic utilization of order sets across many hospitals, effectiveness of evidence-based CPOE ordering practices has been insufficiently explored in community care settings. This study quantitatively analyzed effectiveness of evidence-based CPOE ordering practices for pneumonia patients, measured by mortality, 30-day readmissions, and length of stay health outcomes. The study demonstrates a potentially strong correlation between evidence-based CPOE ordering practices and health outcomes from treating pneumonia. We find that the utilization of order sets to prescribe medications in these cases is beneficial and serves as a sufficient starting point for warranting physician participation in further studies, increasing utilization of the order sets in hospitals, and initiating more narrow focused studies that allow for greater variable control and more granular data collection. Much of the literature indicates relatively little use of the evidence-based medication prescription practices embedded in CPOE design. Yet, some 54% of the U.S. hospitals have implemented CPOE by 2014 [24], and it is likely that many of these hospitals have capabilities for utilization of order sets as one of the core CPOE components and/or already employ pneumonia sets in their clinical workflows.

A study of this size can serve as a catalyst to encourage further collaboration between physicians and clinical informatics researchers in an effort to identify effective applications for CPOE-enabled tools grounded in methodology of evidence-based practices. Our findings could positively influence patient safety and lead to healthcare cost reductions in many areas. Improvements in these areas are important to healthcare facilities that are reviewing their evidence-based practices, CPOE installation and utilization guidelines, and order set governance efforts. These efforts could lead to establishment of new policies governing application of evidence-based practices via wider utilization of technology-enabled clinical workflows, with eventual support for utilization of standardized CPOE ordering practices as a matter of national healthcare policy. This study could also encourage additional research investigations using more granular data to include such outcome variables as hospital infection rates and mediating variables as source of admission and unit where patients were treated. An observational or an experimental study could also help track the exact elements selected or deselected in the sets to avoid such common conditions as allergies to certain medications.