Assessment of level of care recommendations and nursing acuity scores following an appropriateness of care intervention.

BACKGROUND: As part of a multiprong intervention to eliminate waste in cost of hospital accommodations, the InterQual Level of Care (LOC) criteria was deployed by our institution to assign patients to one of three LOCs: acute care, intermediate intensive care unit (IICU) or intensive care unit (ICU). In that intervention, which sought to decrease the number of patients in a higher LOC than what was clinically necessary, patient safety balancing metrics were stable. However, nursing workload, a key balancing metric, has yet to be examined. In this study, we examine nursing workload before and after the intervention using a proprietary nursing acuity score.
METHODS: A retrospective study was conducted analysing admissions at the study institution. Patient's LOC recommendation (as determined by InterQual), assigned (actual) LOC and nursing acuity scores were collected and analysed. Average nursing acuity scores were compared across patients whose InterQual recommendation aligned with actual LOC ('Acute Match' or 'IICU Match') versus patients who were recommended to be in acute care but were receiving IICU care ('Mismatch').
RESULTS: Following the intervention, the per cent of patients in the Mismatch cohort decreased from 13% to 7%. Prior to the intervention, average nursing acuity score for the Mismatch cohort was less than the IICU Match cohort and greater than Acute Match cohort in all departments analysed. After the intervention period, average acuity score in the Mismatch cohort exceeded that of the Acute Match cohort in all eight departments, but the Mismatch cohort's scores differed from the IICU Match cohort in only one department.
CONCLUSION: Collectively, this study demonstrates that our intervention successfully decreased inappropriate use of the IICU LOC, and that the residual Mismatch cohort is a distinct entity, with nursing needs that exceed that of the Acute Match cohort. Thus, a higher LOC can be justified. This demonstrates that a nursing workload metric such as the nursing acuity score can be a valuable complement to clinical criteria such as the InterQual LOC criteria to objectively determine patient's true, necessary LOC and ensure that nursing staff feels adequately staffed to care for patients. © Author(s) (or their employer(s)) 2022. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Suboptimal nursing workload has been linked to poor safety outcomes and increased mortality. However, balancing effects of quality improvement (QI) initiatives on nursing workload have rarely been reported in the QI literature.

This study validates the results of an intervention aimed at reducing inappropriate use of intermediate intensive care unit level of care (LOC) and demonstrates stable nursing workload before and after the intervention. This study shows that a nursing acuity score can be an objective, complementary tool to assess nursing workload in relation to patient’s LOC.

This study underscores the importance of evaluating balancing measures—in this case, nursing workload—as a predictor of sustainability of QI intervention. QI practitioners ought to collaborate with multidisciplinary teams to devise unique tools, such as a nursing acuity score in our study, to capture balancing effects of any intervention deployed.

Background

Addressing the Triple Aim is a commonly shared goal across healthcare institutions. In 2017, our institution deployed an acuity-adaptable unit (AAU) model in an effort to improve two of the three tenets of the Triple Aim: improve quality of care and improve patient’s experience.1 2 An AAU is a care model, wherein the patient remains in the same room throughout their hospitalisation, regardless of whether they are in an intermediate intensive care unit (IICU) level of care (LOC) or an acute care (‘step down’) unit.3 While studies have demonstrated AAUs improve patient safety, length of stay and patient satisfaction outcomes, at our institution, following the deployment of the AAU model, a US $13 million increase in the cost of accommodations was identified.2–5 The increase in costs was driven by an increase in supply of IICU accommodations, as every room in the hospital now had the opportunity to serve as an IICU room. This increased supply led to increased misuse of the IICU LOC, creating waste in the system.5 To address this observation, our institution deployed a three-prong intervention centred on physician engagement and interprofessional collaboration. The purpose of the intervention was to decrease misuse of the IICU LOC as part of a novel programme called the Cost Savings Reinvestment Program, which we have previously described.5 6 Specifically, this intervention involved:

Identification and engagement of physician champions within participating departments and divisions to promote appropriate IICU use.

A set of visual changes to essential electronic medical record (EMR) tools, including a best practice alert to promote daily assessments of appropriate IICU use, modification of the admit order to include LOC definitions, and modification of the patient list system such that LOC was automatically pulled to the patient lists to promote visibility as a checklist.

Data-driven feedback to physician champions, including metrics such as: average hours on IICU per patient, per cent of patients discharged on IICU, etc. Physicians additionally received data to monitor balancing measures.

This intervention, which was deployed in April 2018, increased appropriate IICU use from 50% to 80% and yielded US$5.7 million in savings in the first year across 18 departments and divisions.5 Key patient safety balancing measures, including fall rates, mortality index and number of rapid response team calls, were stable throughout the intervention implementation.5

At our institution, inappropriate accommodation use has been defined as having a patient assigned to a higher LOC than is recommended by the McKesson InterQual Level of Care Criteria 2017.2, which is reviewed by case managers on a daily basis. The InterQual criterion is a clinical decision support tool that makes recommendations on LOC in a condition-specific and patient-specific manner. It uses severity of illness, intensity of service, response to treatment and comorbidities to make recommendations on appropriate LOC.7 8 However, inappropriate IICU occurred due to a variety of reasons, including no case manager input in the emergency department when LOC orders were made, LOC not being discussed by all stakeholders (including physicians, nurses and case managers) and inconsistent LOC documentation.5 At our institution, the recommended LOC is used by nursing leadership to inform staffing, and a patient’s assigned LOC determines the nurse-to-patient ratio. In the ICU, nurse to patient ratio is 1:1–2, in IICU 1:3, and in Acute Care 1:4–5.

The aforementioned intervention changed the average patient load by nurse vis-à-vis decreasing inappropriate IICU use. Prior to the intervention, a greater number of nurses had a 1:3 (IICU) nurse-to-patient ratio, but by decreasing inappropriate IICU use, more nurses had a 1:4–5 nurse-to-patient ratio, corresponding with Acute LOC. As such, we were interested in examining how the change in patient assignment from IICU to Acute Care translates to a change in nursing workload.

There are many approaches to determining nursing workload and, subsequently, nursing staffing. A recent scoping review by Griffiths and colleagues identifies different methods for estimating workload.9 These include patient prototype approaches, where workload is estimated based on disease (eg, using diagnosis-related groups), level of acuity or dependency (eg, as with the Safer Nursing Care Tool10) or a task-based system, where workload is estimated based on time required to complete tasks.11 At our institution, a nursing acuity score is used as a proxy for nursing workload. This score, most similar to the task-based system described by Griffiths and others, is calculated through a proprietary EMR algorithm that takes into account orders in nine domains of patient care: medication orders, assessments, communication needs, admission and transfers, orders, lines/drains/airways, wound care needs, activities of daily living and discharge orders. Scores are calculated and updated every 4 hours, and are available to all staff via a report generated on the EMR. A higher the score indicates a higher nursing workload for that patient.

Understanding how the nursing acuity score varies by LOC is an important balancing metric to the original intervention, as to ensure that nursing staffing is adequate for patients’ needs. Furthermore, both actual workload and perceived workload are associated with job satisfaction and may result in higher turnover rates.9 12–15 Thus, understanding patterns of nursing workload via nursing acuity score by LOC will be critical to inform the sustainability and expansion of this intervention across other departments and other institutions. The objective of this study was to analyse nursing workload via the nurse acuity score, a key balancing measure, before and after the intervention period in participating departments to inform the sustainability of this intervention.

Methods

Setting and data collection

The LOC intervention was deployed in April 2018, with the baseline period taking place from April 2017 through March 2018 (figure 1). Intervention implementation took place from April 2018 through March 2019 and the postintervention period began in April 2019. The postintervention period was defined as April 2019 through March 2020, which is the focus of this balancing measure analysis. Hospital admissions from the baseline period and the postintervention period were collected for eight participating departments and divisions. Of these eight, four were medicine services (Hospital Medicine, Cardiovascular Medicine, Oncology and Pulmonary and Critical Care) and four were surgical services (General Surgery, Neurosurgery, Orthopedic Surgery and Otolaryngology/Head and Neck Surgery). Individual patients’ hospital admission days were categorised into one of three cohorts: Acute Match, IICU Match and Mismatch. Acute Match was defined by having an InterQual recommendation for Acute Care and an actual LOC of Acute Care, IICU Match was defined by having an InterQual recommendation for IICU Care and an actual LOC of IICU Care, and, finally, Mismatch was defined by having an Acute Care or Not Met InterQual recommendation but receiving IICU care (table 1). Patients receiving ICU care were excluded from this study and intervention. Finally, for each hospital admission day, the corresponding nursing acuity score was collected. The university institutional review board waived review for this study based on its classification as quality improvement.

Figure 1

Intervention and study timeline (created by the authors). IICU, intermediate intensive care unit. EMR, electronic medical record; IICU, intermediate intensive care unit.

Table 1

Patient classification

Actual LOC	InterQual recommendation	Cohhort assignment
Acute care	Acute care	Acute match
IICU care	Acute care	Mismatch
IICU care	Not met	Mismatch
IICU care	IICU care	IICU match

Patients who were assigned to critical care (ICU) were not included in this analysis (created by the authors).

ICU, intensive care unit; IICU, intermediate ICU; LOC, level of care.

Patient and public involvement

Patients were not directly involved in the data collection process of this study. All data were anonymised and patient attributes—including their age, race, insurance status—were omitted from analysis. Despite no direct involvement from patients in data collection, the research question of interest specifically addresses patient safety concerns. Suboptimal nursing workload has been linked to poor safety outcomes, thus understanding the sustainability and viability of this intervention via studying workload is a critical metric for patients.16

Data analysis

All data were analysed using Stata V.16.1 (StataCorp LLC, College Station, Texas).17 Continuous data were represented with means and were compared using Wilcoxon rank-sum tests. Categorical data were compared using Fisher’s exact tests. All p values less than 0.05 were considered statistically significant.

Results

Mismatch rate

The mismatch rate was calculated using the proportion of days by department or division spent in the Mismatch Cohort over total patient days. The total mismatch rate at baseline for the eight departments was 13% and 7% in the postintervention period (p<0.0001). Each individual department had a documented decrease in mismatch rate, with statistically significant differences noted in six of the eight departments analysed (table 2).

Table 2

Mismatch rates are the proportion of total patient days spent in the mismatch cohort out of all patient days (mismatch cohort plus Acute Care plus IICU Care) for a given department in the specified time period (created by the authors).

Department		Baseline mismatch rate	Post-intervention mismatch rate	Change in % mismatch patients	P value
Medical services	Hospital Medicine	18%	10%	−8	<0.0001
	Cardiovascular Medicine	25%	16%	−9	<0.0001
	Pulmonary and Critical Care	22%	9%	−13	<0.0001
	Oncology	8%	5%	−3	<0.0001
Surgical services	General Surgery	7%	4%	−3%	<0.0001
	Neurosurgery	10%	8%	−2	0.0838
	Orthopaedic Surgery	3%	1%	−2	<0.0001
	Otolaryngology/Head and Neck Surgery	17%	12%	−5	0.0902
Total		13%	7%	−6	<0.0001

Boldface indicates statistically significant values at 0.05 level.

IICU, intermediate intensive care unit.

Nursing acuity scores by LOC

For all departments in the baseline period, IICU Match had the highest average nursing workload, as defined by the average total nursing acuity score, while Acute Match had the lowest nursing workload (figure 2A). For medicine services, average IICU Match nursing acuity ranged from 44.0 to 62.0, whereas Acute Match nursing acuity ranged from 38.0 to 40.4. In surgical services, baseline workload for IICU and Acute Match cohorts ranged from 57.9 to 73.1 and 42.2 to 56.7, respectively. For medicine services, the Mismatch Cohort average nursing acuity score ranged from 42.7 to 54.4, while in surgical services, the Mismatch Cohort average nursing acuity score ranged from 53.6 to 67.7. In both medicine and surgical services, the average Mismatch nursing acuity score was greater than the Acute Match score but less than the IICU Match score. The Mismatch cohort average acuity score was statistically significantly different (lower) than the IICU Match cohort of the Hospital Medicine, Pulmonary and Critical Care and General Surgery departments. On the other hand, at baseline, the Mismatch cohort was statistically significantly different (greater) from Acute Match in all departments except for Cardiovascular Medicine.

Figure 2

Average total acuity scores were calculated for IICU Match, Mismatch and Acute Match cohorts by department in the baseline (A) and post-intervention (B) time periods. *Indicates p value <0.05. Medicine services refer to Hospital Medicine, Cardiovascular Medicine, Pulmonary and Critical Care and Oncology. Surgical services refer to General Surgery, Neurosurgery, Orthopedic Surgery and Otolaryngology/Head and Neck Surgery. IICU, intermediate intensive care unit.

In the postintervention period, the Acute Match cohort had the lowest average nursing acuity score, ranging from 44.1 to 48.0 in medicine services and from 52.0 to 58.8 in surgical services (figure 2B). In all but three services, IICU Match nursing acuity scores exceeded the Mismatch and Acute Match cohorts. In the three services where the Mismatch cohort exceeded IICU Match—Cardiovascular Medicine, Orthopaedic Surgery and Otolaryngology/Headand Neck Surgery—the differences between IICU Match and Mismatch average nursing acuity score were not statistically significantly different. Furthermore, in the postintervention period, the Mismatch cohort nursing acuity scores were statistically significantly different that the Acute Match cohort (greater) across all eight services and statistically significantly different (lower) than the IICU Match cohort for the Hospital Medicine department. In summary, the Mismatch cohort had nursing acuity scores that exceeded the Acute Match cohort in all departments, and these differences were statistically significant. The Mismatch cohort scores were greater than the IICU Match cohort in three of eight departments and less than the IICU Match in five of eight departments.

Discussion

In this study, we assessed nursing workload—a key balancing measure—following the deployment of an intervention aimed at decreasing inappropriate use of the IICU LOC. We first demonstrated that the intervention was effective at decreasing inappropriate use of the IICU LOC (‘Mismatch cohort’) from 13% to 7% across the eight departments reported in this study between the baseline and postintervention period, confirming our previously reported findings.5

Beyond achieving our desired outcome of decreasing inappropriate IICU LOC use, our analysis of nursing acuity scores demonstrates that at both the baseline and postintervention periods, the Mismatch cohort is a distinct group with nursing workload needs that exceed the Acute Care cohort. Thus, while this intervention decreased the number of patient days spent in the Mismatch cohort, those who remain in the Mismatch cohort warrant a higher LOC with a lower nursing ratio, as their needs are higher than the Acute Care, as demonstrated by the statistically significantly higher nursing acuity scores. In other words, the nursing workload associated with patients in the Mismatch period exceeds that of the Acute Care cohort, as demonstrated by their nursing acuity score. Moreover, in the postintervention period, the Mismatch group nursing acuity score was not statistically significantly different than the IICU Match group in seven of the eight departments. This suggests that the needs of the Mismatch group are not necessarily lower than the IICU Match cohort, and, thus, can benefit from a similar nursing staffing ratio as in the IICU Match cohort.

Together, this study demonstrates that, first, relative nursing workload, as measured by nursing acuity scores across LOCs within a department, was stable following the intervention. Second, the patients remaining in the Mismatch group after the intervention was deployed have needs that are greater than the Acute Match cohort, and, thus, a higher accommodation is warranted. Had the postintervention period showed an increased nursing acuity score in the Acute Match cohort relative to the Mismatch or IICU Match cohorts, there would be concern that the InterQual criteria were underestimating required workload by assigning patients with high needs to a lower LOC (Acute Care). Had this been the case, there would be concern for poor sustainability of this intervention.

Instead, this study demonstrates that nursing workload should be used alongside the InterQual criteria to best inform the appropriate LOC for a patient. While we found that the InterQual criteria appropriately places patients in the correct cohort in 93% of cases, the remaining 7% require additional clinical judgement to ensure that the patient’s needs are being best met. Furthermore, it demonstrates that the nursing acuity score can be an objective measure to determine whether the patient’s nursing needs are sufficiently high to require a higher LOC than what was recommended from InterQual. Since the nursing acuity score is calculated and reported in the electronic medical record per patient every 4 hours, the score can serve as a dynamic and objective datapoint for nurse managers, registered nurses, physicians and case managers to collectively determine how to reduce waste, while ensuring that nursing staff has sufficient bandwidth to meet all patients’ needs.

One strength of the aforementioned LOC intervention, and subsequent implementation and evaluation, was the interdisciplinary collaboration of stakeholders. Sponsored by the chief medical officer of our institution, the Cost Savings Reinvestment Program enables frontline providers to propose value creation initiatives, with savings reinvested into other clinical initiatives.6 This study specifically relied on engagement from nurses, nursing leadership, case managers and physicians to ensure that the LOC intervention continued to be adhered to in the postintervention period.

Limitations

The objective of this study was to determine how nursing workload changed as a result of the aforementioned intervention. While we used the nursing acuity score as a quantitative metric for our analysis, there are elements of nursing workload that cannot be captured numerically. For example, the concept of environmental turbulence—that is, the interaction between nurses and their working environment, wherein there is instability and rapid changes—has been well described as a source of increase burnout, exhaustion and poor safety outcomes.18 19 However, the nursing acuity score does not take into account environmental turbulence that may negatively affect working conditions as this intervention was deployed. Moreover, a qualitative lens, including focus groups and interviews, could help contextualise these findings and provide greater insights as far as barriers to sustainability of this intervention. For instance, interviews or focus groups could reveal additional nursing education and training that was or is required (eg, for travel nurses new to this protocol and AAU model), decreased productivity due to confusion over the intervention implementation and so forth. Finally, the original intervention, and, thus, subsequent analysis, was observational in nature. A randomised, controlled approach would have allowed for stronger conclusions to be drawn. Limitations of the intervention itself have been previously described5 and included the fact that this was conducted in a single institution and that only one LOC determination tool (InterQual criteria) was used.

Conclusion

In this study, we demonstrated that the deployment of an appropriateness of LOC intervention did not inadvertentlyinadverdently increase nursing workload, a key balancing measure. We show that patients who continue to be placed in an inappropriate LOC have nursing workloads that are statistically significantly higher than their acute care counterparts, thus a higher LOC can be justified. Importantly, our study demonstrated that while the InterQual tool was effective in placing patients in the right LOC cohort for 93% of patient days, the remaining 7% require additional clinical judgement, and the nursing acuity score can be an objective way to make that determination. Healthcare institutions seeking to eliminate waste via reduction of inappropriate IICU accommodation use should include nursing workload along with clinical criteria, such as the InterQual tool, to ground decisions made by the entire care team to best care for patients.