Unexpected clinical outcomes following the implementation of a standardised order set for hepatic encephalopathy.

OBJECTIVE: We evaluated the effect on clinical outcomes of implementing a standardised inpatient order set for patients admitted with hepatic encephalopathy (HE).
METHODS: A retrospective review of patients with cirrhosis admitted with HE. Hospital admissions for HE for which the electronic health record (EHR) order set was used were compared with admissions where the order set was not used. Primary outcome was length of hospital stay (LOS). Secondary outcomes were 30-day readmissions, in-hospital complications, in-hospital and 90-day mortality.
RESULTS: There were 341 patients with 980 admissions over the study period: 263 patients with 736 admissions where the order set was implemented, and 78 patients with 244 admissions where the order set was not implemented. Median LOS was 4 days (IQR 3-8) in the order set group compared with 3 days (IQR 2-7) (p<0.001); incidence rate ratio 1.37 (95% CI 1.20 to 1.57), p<0.001. 30-day readmissions rate was 56% in the order set group compared with 40%, p=0.01; OR for readmission was 1.88 (95% CI 1.04 to 3.43), p=0.04. Hypokalaemia occurred in 46% of admissions with order set use compared with 36%, when the order set was not used; p=0.003, OR 1.72 (95% CI 1.22 to 2.43), p=0.002. No significant differences were seen for in-hospital mortality and 90-day mortality.
CONCLUSION: Implementation of an inpatient EHR order set for use in patients with HE was associated with unexpected clinical outcomes including increased LOS and readmissions. The convenience and advantages of standardisation of patient care should be balanced with a degree of individualisation, particularly in the care of medically complex patients. Furthermore, standardised processes should be evaluated frequently after implementation to assess for unintended consequences. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Standardised order sets are being increasingly used in delivery of care in hospitalised patients, but data are limited on their impact on patients with cirrhosis.

The implementation of a standardised electronic health record order set for patients admitted with hepatic encephalopathy at our institution was associated with unexpected clinical outcomes including increased hospital length of stay and increased 30-day readmissions.

The results from our study illustrate the challenges in standardising the care of medically complex patients in general, where good clinical judgement is irreplaceable to ensure successful clinical outcomes.

Introduction

Hepatic encephalopathy (HE) is a common complication of liver disease and patients with HE have high rates of healthcare utilisation.1 Several studies have looked at the delivery of quality care to hospitalised patients with cirrhosis and have concluded that a low proportion of patients receive high quality care.2–5 In 2010, a set of quality indicators were derived by an expert panel with the intention of setting a platform for the delivery of quality care to patients with cirrhosis.6

Electronic health records (EHRs) are now almost ubiquitous in the American healthcare systems. Order sets are clinical decision support tools that aim to help providers by reducing variability, ordering pre-specified doses of medications and promoting adherence to guidelines in the care of patients, often who have a specific condition or disease. The use of order sets has been proposed as one approach to improve quality care in cirrhosis.5 Standardised order sets have been shown to improve adherence to clinical guidelines in venous thromboembolism and stroke,7 8 and improve clinical outcomes in the management of myocardial infarction and sepsis.9 10 One recent study looked at the development of an EHR order set to improve clinical outcomes in patients with variceal bleeding and demonstrated improvements in both the use and time to administration of antibiotics.11

Even though there is clear evidence of the benefits of order set use, there are also cautionary examples of their potential drawbacks on patient outcomes and healthcare utilisation. One study showed that a higher proportion of patients were exposed to increased risk of harm following the implementation of a standardised order set for venous thromboembolism prophylaxis.12 Overuse of medications such as sleeping aids, inappropriate long-term proton-pump inhibitor use and overuse of telemetry in the hospitalised setting have also been identified as unintended consequences of order set use.13–15

The effect of order set use on clinical outcomes on patients with cirrhosis and HE is unknown. We aimed to evaluate utilisation of an inpatient order set derived for use in patients with cirrhosis admitted to the hospital with HE and to evaluate the impact on clinical outcomes.

Methods

Setting

The University of Minnesota Medical Center (UMMC) is a tertiary care medical centre with a high-volume liver transplant programme in the upper mid-western USA. Depending on patient acuity, patients with cirrhosis are admitted to either a Hospital Medicine team—staffed by a hospitalist with or without internal medicine residents—or the intensive care unit (ICU). An EHR (EPIC, Verona, Wisconsin, USA) was implemented at UMMC for use in the inpatient setting in 2011.

The HE order set was developed and made available for use at UMMC in December 2013. It is available (and encouraged) for use by all providers at any time during an admission and incorporates standardised pre-checked orders for the management of HE including a nurse-driven lactulose protocol, which involves nursing assessment of a patient’s mental status based on West Haven criteria.16 The order set includes a brief summary of stages of HE and frequent nursing assessment of HE severity. The presence of HE results in administration of 20 g lactulose orally, or 100 g lactulose rectally, every 2 hours as needed until resolution of symptoms. The content of this order set is consistent with evidence-based, standard practices in the management of HE.16

In addition, the order set also includes pre-checked orders for laboratory and medication orders for rifaximin, as well as options to order radiology studies and electrolyte replacement (figure 1). Of note, this order set was developed and implemented prior to conceptualisation and conduction of this study. This study was conducted with the approval of the University of Minnesota Institutional Review Board.

Figure 1

Hepatic encephalopathy order set.

Study design

We performed a single centre, retrospective review of EHR data of the patients with cirrhosis admitted to UMMC with HE from December 2013 to December 2017. Hospital admissions for which the EHR order set was used at hospital admission were compared with admissions where the order set was not used.

Study outcomes

The primary outcome was hospital length of stay (LOS). Secondary outcomes were 30-day readmissions, in-hospital complications, in-hospital mortality and 90-day mortality. In-hospital complications included hypernatraemia (defined as serum sodium >145 mEq/L or an increase in serum sodium >5 mEq/L at any point in the hospital admission); hypokalaemia (defined as serum potassium <3.5 mEq/L at any point in the admission) and acute kidney injury (defined as an increase in serum creatinine >0.3 mg/dL over 48 hours).

Study population

Patients were included in the study if they were admitted to the UMMC Hospital Medicine service with an International Classification of Diseases-Ninth Revision (ICD-9) or ICD-10 code for cirrhosis and HE or altered mental status in admission or discharge coding. All admissions for HE were validated by manual chart review by a single physician in accordance with the definition of HE provided in the current American Association for the Study of Liver Diseases practice guideline.16 Patients with a diagnosis code for HE that did not meet criteria for HE after chart review were excluded from this study.

Patients were also excluded if they were less than 18 years old; if they had previously received a liver transplant; if they were originally admitted for non-HE condition; if they were admitted to the ICU or any other service than Hospital Medicine; if they were admitted for another acute neurological condition (cerebrovascular accident, meningitis, acute intoxication, intracranial haemorrhage or seizure) or if they were placed on comfort care/hospice care during their hospitalisation.

Data acquisition

Patient demographics (age, gender, ethnicity); hospitalisation data (admission and discharge dates, hospital readmission data, hospital admission unit); laboratory values; medications; co-morbid conditions and order-set utilisation data were obtained from the University of Minnesota Clinical and Translational Science Institute’s clinical data repository via their Informatics Consulting Service. Aetiology of liver disease and precipitating factor for HE was obtained by manual chart review by a single physician.

Statistical analysis

Descriptive statistics were calculated and presented by group. Mean and SD were presented for continuous variables that are normally distributed. Median and IQR were presented for length of stay and number of readmissions as their distribution was skewed. Frequency and percentage were presented for categorical variables.

Bivariate analysis was conducted to compare order set use versus standard care using two-sample t-test for continuous variables that were normally distributed, non-parametric Wilcoxon test for length of stay and number of readmissions and χ2 test (or Fisher’s exact test if any cell count was less than 5) for categorical variables. Multivariate analysis was performed to examine the effect of order set use on patient outcomes, adjusting for patient demographics including age at first HE episode, gender, race, aetiology of liver disease, rifaximin use and precipitant for HE. These covariates were selected based on scientific rationale as clinical factors known to affect clinical outcomes in patients with HE. Negative binomial regression models were conducted for length of stay and number of readmissions, and incidence rate ratio (IRR) was presented. Logistic regression model was conducted for binary outcomes, and OR was presented. Analyses were performed in SAS V.9.4 (SAS Institute). P values <0.05 were considered statistically significant.

Results

Baseline characteristics

There were a total of 341 unique patients with 980 admissions over the study period: 263 unique patients with 736 admissions where the order set was implemented, and 78 unique patients with 244 admissions where the order set was not implemented. Patients in the group where the order set was not used were older compared with the group where the order set was used (mean age 58.9 years vs 56.0 years, p=0.05). Alcohol-related cirrhosis was the most common aetiology of liver disease in both groups. The mean model of end-stage liver disease (MELD)/MELD-Na was 20.9/20.5 for the entire cohort. No significant differences were observed between the two groups for gender, ethnicity, mean MELD/MELD-Na score, co-morbidities, aetiology of liver disease or precipitant of HE. Rifaximin was used in 97% of patient encounters when order set was used compared with 83% when the order set was not used, p<0.001. The most common documented HE precipitant was medication non-adherence. Despite extensive manual chart review, a large number of admissions had an unknown precipitant (table 1).

Table 1

Baseline characteristics

	Order-set usedn=263	Order-set not usedn=78	P value
Female, n (%)	108 (41)	27 (35)	0.31
Caucasian race, n (%)	204 (84)	64 (86)	0.55
Age, mean (SD)	56.0 (10.5)	58.9 (11.3)	0.05
MELD, mean (SD)	20.7 (8.5)	21.0 (7.2)	0.79
MELD-Na, mean (SD)	19.7 (10.2)	21.3 (8.4%)	0.20
Aetiology of liver disease			0.22
Alcohol, n (%)	113 (45)	27 (35)
Hepatitis C, n (%)	28 (11)	14 (18)
Alcohol and hepatitis C, n (%)	28 (11)	7 (9)
Other, n (%)	81 (32)	29 (38)
Comorbid conditions
CKD, n (%)	60 (23)	12 (15)	0.16
ESRD, n (%)	14 (5)	4 (5)	1.00
HIV, n (%)	1 (0.4)	1 (1)	0.41
Diabetes, n (%)	104 (40)	27 (35)	0.43
COPD, n (%)	20 (8)	8 (10)	0.45
Coronary artery disease, n (%)	21 (8)	7 (9)	0.78
No. of hospital encounters, n	736	244
Rifaximin use, n (%)	712 (97)	203 (83)	<0.001
Precipitant of HE			0.13
Medication non-adherence, n (%)	247 (34)	80 (33)
Unknown precipitant, n (%)	128 (17)	64 (23)
GI bleed, n (%)	33 (4)	13 (5)
Urinary tract infection, n (%)	76 (10)	13 (5)
Spontaneous bacterial peritonitis, n (%)	44 (6)	13 (5)
Other, n (%)	208 (28)	70 (29)

CKD, chronic kidney disease; COPD, chronic obstructive pulmonary disease; ESRD, end-stage renal disease; GI, Gastrointestinal; HE, hepatic encephalopathy; MELD, model of end-stage liver disease.

Primary outcome

The median LOS for admissions where the order set was used was 4 days (IQR 3–8 days) compared with 3 days (IQR 2–7 days) for admissions where the order set was not used (p<0.001). After adjustment for age, gender, race, aetiology of liver disease, rifaximin use and HE precipitant, the incidence rate for increased LOS for the order set use group was 37% higher than the incidence rate for the group which did not use the order set (IRR 1.37 (95% CI 1.20 to 1.57), p<0.001) (tables 2 and 3).

Secondary outcomes

Thirty-day readmissions were more common in the order set group, 56% vs 40%, p=0.01. The median number of readmissions was 1 (IQR 0–3) for admissions where the order set was used compared with 0 (IQR 0–1) when the order set was not used (p=0.001). After adjustment for age, gender, race, aetiology of liver disease, rifaximin use and HE precipitant, the OR for readmission in patients within 30 days for whom the order set was used was 1.88 (95% CI 1.04 to 3.43), p=0.04 (tables 2 and 3).

With regard to in-hospital complications, hypernatraemia occurred in 51% of admissions where the order set was used compared with 44% when the order set was not used, p=0.05; OR 1.19 (95% CI 0.86 to 1.66), p=0.11, on multivariate analysis. Hypokalaemia occurred in 46% of admissions using the order set compared with 36% of admissions that did not use the order set, p=0.003; OR 1.72 (95% CI 1.22 to 2.43), p=0.002, on multivariate analysis. Acute kidney injury did not vary depending on order set usage, 47% vs 42%, p=0.19; OR 1.31 (95% CI 0.94 to 1.85), p=0.11, on multivariate analysis (tables 2 and 3).

In-hospital mortality was 2% for admissions using the order set compared with 1% for those not using the order set but this did not reach statistical significance, p=1. Multivariate analysis was unable to be performed for in-hospital mortality due to the low number of deaths. Ninety-day mortality was 35% for admissions using the order set compared with 29% for admissions not using the order set, p=0.34; OR 1.02 (95% CI 0.55 to 1.89), p=0.95, on multivariate analysis (tables 2 and 3).

Discussion

HE is a common complication of cirrhosis for which hospital admissions are increasing, making it an ideal target for the implementation of a standardised order set.17 However, the pathogenesis of HE is complex and remains poorly understood—and in our study, poorly documented. HE is a syndrome with several causes including infection and medication non-adherence; a recent study showed that 40%–76% of patients admitted with HE had multiple concurrent precipitants.18 With the fragility of patients with decompensated cirrhosis in general, careful assessment and management of all patients admitted to hospital with HE is essential.

Standardised EHR order sets have been proposed as one solution to address the current deficit in quality care delivered to patients with cirrhosis.2–5 In our study, despite high utilisation of an inpatient EHR order set for HE, order set use was associated with increased hospital LOS and 30-day readmissions in this study. These findings persisted on multivariate analysis and demonstrate the difficulty in adopting standardised order sets in the care of medically complex patients. It does not appear that patients for whom providers opted to use the order set were significantly clinically different than those patients in whom the order set was not used.

Adherence to standardised order sets can have unintended outcomes in patient care.12–15 19 Our findings contribute to an emerging body of literature describing the limitations of order set use and demonstrate the importance of critical thinking in patient care, particularly in a patient population as susceptible to harm as those with cirrhosis.20 HE is a syndrome—not a single entity—and careful, expert attention to the underlying precipitants as well as the recognition of the potential adverse effects of any intervention is fundamental. Certainly, medication non-adherence is a common precipitant—and lactulose is required in majority of patients admitted with HE. However, a diagnostic and treatment approach that lacks nuanced tailoring to a specific patient’s presentation can be problematic. While order sets can reduce unnecessary variation in care, a degree of individualisation may be necessary in the management of these complex patients; in fact, order sets could be modified to encourage such assessment and individualisation.

One possible explanation for our findings is that the increased rate of complications observed in the order set group would logically lead to both increased hospital LOS and 30-day readmissions. Order set use was significantly associated with hypokalaemia—a known side effect of lactulose overuse—which may appear innocuous at first glance. However, hypokalaemia has been shown elsewhere to increase hospital and ICU lengths of stay in patients with HE.21 Moreover, hypokalaemia is associated with increased all-cause mortality in the elderly and in patients with chronic kidney disease (CKD).22 23 We also observed trends for increased rates of hypernatremia and acute kidney injury, both of which are more traditionally associated with poor clinical outcomes in patients with cirrhosis.24 25 There are multiple possible explanations for these findings, but we posit that patients treated with the order set were treated more aggressively with lactulose. Overuse of diuretics may have contributed to the increased rates of hypokalaemia and acute kidney injury seen in the order set group—and indeed may even account for a significant proportion of the ‘unknown’ HE precipitants in the whole cohort—but would not explain the higher rates of hypernatraemia. Administration of lactulose is the foundation of the treatment of HE but can have significant side effects including hypovolaemia, hypokalaemia and metabolic acidosis. It is possible that our order set was too aggressive in the administration of lactulose, particularly given the high rate of comorbidities (especially CKD and diabetes) in our patient population. Our outcomes may also have been a reflection of the high mean MELD-Na score at admission in our cohort, which in itself has been associated with higher readmission rates, although a significant difference was not observed between the two study groups.26–28

On a positive note, we observed higher rates of rifaximin use in the order set group. Although not a specific outcome of our study, rates of rifaximin use were high in both groups but significantly higher in the order set group. Rifaximin is a non-absorbable antibiotic that has been shown to reduce the risk of development of HE and improves quality of life outcomes in patients with cirrhosis.29 30 In a recent meta-analysis, the use of rifaximin was associated with reduced overall mortality in patients with HE.31 Despite increased rifaximin use, we did not observe any improvement in mortality in the order set group: this may again be a reflection of higher baseline MELD-Na in our cohort but may also be due to the exclusion of patients admitted to the ICU from our study.

As our study groups were not randomised, it is important to consider why the order set was used in some encounters and not others. Both groups were similar with the exception of slightly older age in the order set group, which was not large enough to account for the difference in outcomes. Reasons for non-utilisation of order sets may include provider preference, lack of awareness of order set and lack of mandatory implementation within a health system.32 33 In this study, it is possible that use of the order set was affected by providers’ perceived loss of control with regard to the nurse-driven lactulose protocol. The remainder of the content of the order set was consistent with standard, evidence-based practices and was unlikely to have contributed to these concerns.

Our study is the first to evaluate the implementation of an EHR order set in patients admitted to hospital with HE. Our findings are important due to their unexpected nature and illustrate the difficulty encountered in improving the quality of care in patients with decompensated cirrhosis. Hospital length of stay and 30-day readmissions are well-described metrics for resource utilisation which have significant financial implications.34 35 Increased hospital length of stay and readmissions have been shown both to increase the cost of care and worsen clinical outcomes in patients with decompensated cirrhosis.26 36 This is particularly relevant at a time when healthcare costs are spiralling out of control in the USA: hospitalisation costs for patients with cirrhosis increased by 30% to $7.37 billion in 2014.37

The strengths of our study include a large number of patients and patient encounters. Manual review of our data ensured accuracy and granularity, particularly with regard to HE precipitant which can be difficult to elucidate with automated data collection. The 4-year study period also accounted for any lag that can be occur following the implementation of EHR order set due to ‘late adopters’. Limitations of our study include the single centre, retrospective design which may limit the generalisability of our results. Second, it can be difficult to capture all readmissions to a tertiary care referral hospital as patients often re-present to other institutions. Third, a large number of HE precipitants in our study were deemed ‘unknown’ despite extensive evaluation; however, this is a reflection of real-world practice. Finally, we acknowledge that patients with more severe HE at presentation may have preferentially had the order set used in their care. However, it is difficult to fully account for this possibility in our study: the West Haven criteria have significant intra-operator and inter-operator variability, and the mental status of patients with significant liver disease can fluctuate markedly.38 Again, we note that there was no significant difference in baseline liver function, represented by the MELD-Na score on admission, between the two study groups.

In conclusion, the implementation of a standardised EHR order set for patients admitted with HE at our institution was associated with unexpected clinical outcomes including increased hospital LOS and increased 30-day readmissions. The results from our study illustrate the challenges in standardising the care of medically complex patients in general, where good clinical judgement is irreplaceable to ensure successful clinical outcomes. Moreover, attempts to standardise care should be carefully monitored for adverse effects. Further study is required to understand why providers may opt out of using order sets and to optimise the utilisation of EHR to improve the quality of care delivered to patients with cirrhosis in hospital.

Table 2

Unadjusted clinical outcomes comparing patients with and without order set use

	Order set usedn=263	Order set not usedn=78	P value
Total no. of admissions, n	736	244
Length of stay, median (IQR)	4 (3–8)	3 (2–7)	<0.001
No. of readmission within 30 days, median (IQR)	1 (0–3)	0 (0–1)	0.001
Readmitted within 30 days of previous discharge, n (%)	148 (56)	31 (40)	0.01
Complications
No. of encounters with hypernatraemia, n (%)	379 (51)	108 (44)	0.05
No. of encounters with hypokalaemia, n (%)	341 (46)	87 (36)	0.003
No. of encounters with AKI, n (%)	342 (47)	102 (42)	0.19
In-hospital mortality	5 (2%)	1 (1%)	1
90-day mortality	93 (35%)	23 (29%)	0.34

AKI, acute kidney injury.

Table 3

Adjusted risk for clinical outcomes with regard to order set use

		P value
Hospital length of stay	IRR 1.37 (95% CI 1.20 to 1.57)	<0.001
30-day readmissions	OR 1.88 (95% CI 1.04 to 3.43)	0.04
Complications
Hypernatraemia	OR 1.19 (95% CI 0.86 to 1.66)	0.11
Hypokalaemia	OR 1.72 (95% CI 1.22 to 2.43)	0.002
Acute kidney injury	OR 1.31 (95% CI 0.94 to 1.85)	0.11
90-day mortality	OR 1.02 (95% CI 0.55 to 1.89)	0.95

Outcomes adjusted for age, gender, race, aetiology of liver disease, rifaximin use and precipitant for hepatic encephalopathy.

IRR, incidence rate ratio.