Adding value to daily chest X-rays in the ICU through education, restricted daily orders and indication-based prompting.

BACKGROUND: Chest X-rays (CXRs) are traditionally obtained daily in all patients on invasive mechanical ventilation (IMV) in the intensive care unit (ICU). We sought to reduce overutilisation of CXRs obtained in the ICU, using a multifaceted intervention to eliminate automated daily studies.
METHODS: We first educated ICU staff about the low diagnostic yield of automated daily CXRs, then removed the 'daily' option from the electronic health records-based ordering system, and added a query (CXR indicated or not indicated) to the ICU daily rounding checklist to prompt a CXR order when clinically warranted. We built a report from billing codes, focusing on all CXRs obtained on IMV census days in the medical (MICU) and surgical (SICU) ICUs, excluding the day of admission and days that a procedure warranting CXR was performed. This generated the number of CXRs obtained every 1000 'included' ventilator days (IVDs), the latter defined as not having an 'absolute' clinical indication for CXR.
RESULTS: The average monthly number of CXRs on an IVD decreased from 919±90 (95% CI 877 to 963) to 330±87 (95% CI 295 to 354) per 1000 IVDs in the MICU, and from 995±69 (95% CI 947 to 1055) to 649±133 (95% CI 593 to 697) in the SICU. This yielded an estimated 1830 to 2066 CXRs avoided over 2 years and an estimated annual savings of $191 600 to $224 200. There was no increase in reported adverse events.
CONCLUSION: ICUs can safely transition to a higher value strategy of indication-based chest imaging by educating staff, eliminating the 'daily' order option and adding a simplified prompt to avoid missing clinically indicated CXRs.

Introduction

Standard practice in the intensive care unit (ICU) has traditionally included daily chest X-rays (CXRs) in all patients receiving invasive mechanical ventilation (IMV), under the assumption that daily imaging is necessary to discover occult malposition of endotracheal tubes (ETT) and central venous catheters and identify otherwise undetected conditions, such as pneumonia or pneumothorax.1–3 However, recent data suggest that daily CXRs in ICU patients receiving IMV are of low diagnostic yield and have negligible impact on management decisions.4–8 As is often the case, uniform change in local ICU culture has been slow to adapt to the evidence, and the elimination of automated daily CXRs in ICU patients on IMV has yet to be uniformly adopted. Rather than set out to demonstrate the safety of this practice, we sought to use methods of organisational practice change to transform our organisation’s local culture to one that lends greater value9 10 to our practice in the ICU. Based on anecdotal knowledge of local practice, we hypothesised that our adult ICU practice overused daily CXRs, and that a systematic initiative of targeted education and a modified electronic ordering system could lead to a significant reduction in potentially unnecessary CXRs. We speculated this would in turn result in significantly added value9 10 to CXRs in our ICU, and significant cost savings to our institution, without any measurable reduction in the quality of ICU care.

We set out to achieve our objective of reducing CXR overutilisation in the ICU through a stepwise approach of (1) assessing current practice, (2) educating staff on the low clinical utility of daily CXRs, (3) eliminating the option of electronic health records (EHR)-based orders for automated daily CXRs and (4) adding a prompt to the daily ICU rounding checklist to query whether a CXR is clinically indicated. The latter step was added to avoid a ‘more than clinically necessary’ reduction in ordering habits and avoid any unintended deficit of important information in patients for whom a CXR may be clinically warranted.

Methods

The University of Vermont Medical Center (UVMMC) is a 400-bed academic tertiary care hospital with a separate medical ICU (MICU) and surgical ICU (SICU), each housing 21 beds. The current investigation was approved by the ’High Value Care' programme within the Department of Medicine at UVMMC. The project was reviewed and exempted by the UVM Institutional Review Board as a quality improvement project involving nonhuman research subjects’ data. Data analytics from the James Jeffords Institute of Quality established a retrospective baseline measurement of daily CXR practice in patients receiving IMV in both ICUs over the preceding 12 months. Because we were most interested in reducing the number of potentially ’unnecessary' CXRs, we focused on all patient census days associated with IMV, excluding the day of admission or intubation or any day involving a procedure that warranted follow-up CXR (thoracentesis, ETT placement, central line placement or chest tube placement). This led to a denominator of all ’included ventilator days' (IVDs) that did not have an admission or procedural-related ’standard of care' indication for a CXR on that census day. IVDs were compiled from billing records as all days billed for IMV, minus all admission days and days during which a procedure warranting a CXR was billed. The numerator was the monthly total number of CXRs obtained on an IVD (obtained from the same billing database), and the denominator was monthly IVDs divided by 1000, leaving a metric of ‘CXRs ordered per 1000 IVDs’.

Because we expected that a CXR would be clinically indicated on several non-admission, non-procedural days (eg, suspicion for evolving pneumonia, chest tube removal, and so on), we did not expect the numerator to fall to zero, even under high-efficiency conditions. The clinical champions for the project reviewed the baseline data and established a target for improved value. Because appropriate benchmark targets had not been published, nor are likely identical across institutions, the team used anecdotal evidence to conclude on an a priori target of around 300 CXRs per 1000 IVDs for the MICU. Because length of stay (LOS) is shorter in the SICU, with a greater number of patients having chest tubes, the team concluded that the appropriate target rate lay between 300 and 500 (per 1000 IVDs) in the SICU. The chosen interventions centred on education and system-based change. Education occurred between June 2013 and March 2014 and included outreach to all critical care nursing, respiratory and physician staff, radiology technicians responsible for obtaining CXRs in the ICU, and resident physicians during their ICU rotation. System-based changes included removing the ‘daily’ order option from the ‘portable chest’ radiograph order within the EHR, and adding a prompt to the ICU rounding checklist to ask clinicians whether a CXR was clinically indicated that day. Once education had been fully initiated, and system-based changes were completed, monthly performance measures were published to a quality-reporting dashboard and periodically fed back to staff.

In addition, other outcomes thought to potentially be impacted by our intervention were prospectively followed in established quality dashboards, including monthly averages for MICU and SICU LOS and observed to expected (O:E) mortality. ICU LOS was measured internally, and O:E mortality rates were obtained from the University HealthSystem Consortium database. An audit was also made of our event reporting system regarding endotracheal tube misplacement or missed diagnoses over the study period using the search terms: malposition, dislodgement, main stem, advance or advanced ETT or tube.

Estimated cost savings were calculated using an estimated cost of each CXR to the institution, the latter based on both internally generated cost estimates and those published elsewhere.11 We first estimated the quarterly number of ’CXRs saved' by using the equation.

This equation was selected to normalise changes in the volume of CXRs on an IVD to the volume of IVDs. This provided a value of rate reduction, which when multiplied by the actual number of IVDs, generated an estimated number of potentially ’unnecessary CXRs' not obtained each month. We then summed the estimated number of CXRs saved each month over the 2 years following our implementation, and divided by the total number of IVDs over the same 2 years, to yield an average rate of CXRs avoided per 1000 IVDs. 95% confidence limits were calculated for the average rates of CXRs obtained per 1000 IVDs, and for the average rates of CXRs avoided per 1000 IVDs for both the MICU and the SICU.

The upper and lower confidence limits for CXRs avoided per 1000 IVDs were then multiplied by the total number of IVDs to yield upper and lower confidence limits for total number of CXRs avoided. These figures were then multiplied by the estimated unit cost to provide upper and lower confidence limits of cost savings.

Origin software (V.8.5.1, OriginLab, Northampton, Massachusetts, USA) was used for paired t-tests to compare monthly rates of daily CXRs (per 1000 IVDs) over the 12 months before and after our intervention, excluding the 12-month transition phase. Similar comparisons were made between pre-intervention and postintervention monthly averages for ICU LOS and observed to expected (O:E) mortality rates. P values<0.05 were considered clinically significant.

Results

Following our intervention, the average monthly number of CXRs performed in the MICU on non-admission, non-procedural days (IVDs) decreased by 64% from 919±90 (95% CI 877 to 963) per 1000 IVDs during the 12-month period preceding our intervention, to 330±87 (95% CI 295 to 354) per 1000 IVDs during the second 12-month period following intervention (figure 1) (P<0.001). The average monthly number of CXRs performed on an IVD in the SICU decreased by 35%, from 995±69 (95% CI 947 to 1055) to 649±133 (95% CI 593 to 697) per 1000 IVDs following intervention (figure 2) (P<0.001). The average rates of CXRs avoided over the 2 years following our intervention were 492 (95% CI 467 to 517) saved per 1000 IVDs in the MICU and 248 (95% CI 207 to 248) in the SICU. This yielded an estimated range of 1380 to 1529 potentially unnecessary CXRs avoided in the MICU and 450 to 537 CXRs avoided in the SICU over the first 2 years following the intervention.

Figure 1

Monthly calculated figures for number of CXRs ordered on an included ventilation day (IVD), per 1000 IVDs in the MICU (open circles referenced to left-hand vertical axis) for pre-implementation (thick dashed lines demonstrate ±2 SDs), transition and postimplementation years. Thin dashed lines represent linear fits to data for each successive year. Total number of non-admission, non-procedural ventilation days (IVDs) in the MICU is provided by solid grey bars (referenced to right-hand axis). Dates of starting staff education (June 2013) and modifications to ‘daily’ order option and checklist prompt (March 2014) are designated by black arrows on timeline. CXR, chest X-rays; MICU, medical intensive care unit.

Figure 2

Monthly calculated figures for number of chest X rays (CXRs) ordered on an included ventilation day (IVD), per 1000 IVDs in the SICU (open circles referenced to left-hand vertical axis) for pre-implementation (thick dashed lines demonstrate ±2 SDs), transition and postimplementation years. Thin dashed lines represent linear fits to data for each successive year. Total number of non-admission, non-procedural ventilation days (IVDs) in the SICU is provided by solid grey bars (referenced to right-hand axis). Dates of starting staff education (June 2013) and modifications to ‘daily’ order option and checklist prompt (March 2014) are designated by black arrows on timeline. CXR, chest X-rays; SICU, surgical intensive care unit.

Using an institutional estimate of $82 in direct costs, and $94 in indirect costs not directly reimbursed through the patient’s diagnosis-related group, this reduction in CXRs yielded an estimated savings that ranged from $322 000 to $363 600 in total healthcare systems costs over 2 years, slightly over half of which was in unreimbursed costs absorbed by the hospital. However, by the end of the transition year, rates had substantially reduced from their baseline, yielding rates over the final year of 596 (95% CI 556 to 636) CXRs avoided per 1000 IVDs in the MICU and 356 (95% CI 317 to 394) CXRs avoided per 1000 IVDs in the SICU. Working with the most recent 12-month rate of CXRs avoided per 1000 IVDs, practice change led to an annual reduction of approximately 799 to 913 CXRs in the MICU and 290 to 361 CXRs in the SICU, with an estimated annual savings of $191 600 to $224 200.

With respect to undesired consequences, no event reports matching our search terms could be found during the pre-intervention, transition or postintervention periods. Furthermore, there was only one postintervention event report discovered under the subject category of other catheter or tube problem: a central venous catheter unintentionally removed while turning a patient in bed, which was not ‘undiscovered’ due to a forgone daily CXR.

There was no significant difference between pre-intervention and postintervention monthly average LOS in the MICU (3.74±0.45 vs 3.49±0.40 days, P=0.21) or in the SICU (3.16±0.67 vs 3.09±0.30 days, P=0.75) (figure 3). There was a non-significant trend towards an actual decrease in O:E mortality in the MICU (1.17±0.21 vs 1.07±0.14, P=0.09) and in the SICU (1.19±0.34 vs 0.92±0.30 days, P=0.05). Linear regression trend lines were fitted to the plotted monthly values (figure 3).

Figure 3

Monthly values for average LOS in the MICU (solid black circles) and SICU (solid grey circles) are plotted against the entire pre-implementation and postimplementation timeline, along with linear regression fitted plots (thin solid black and grey lines). Monthly values for observed to expected mortality (O:E mortality) for the MICU (solid black line) and SICU (solid grey line) are plotted along the same timeline, along with liner regression fitted plots (black and grey dashed lines). LOS, length of stay; MICU, medical intensive care unit; SICU, surgical intensive care unit.

Discussion

The routine practice of daily CXRs on all patients receiving IMV in the ICU originated from collective expert opinion claiming a high rate of unsuspected findings detected by routine CXRs in this patient population,1–3 and was up until recently supported by the American College of Radiology (ACR).12 13 However, studies in surgical ICUs found low detection rates of clinically significant, unexpected findings, leading some investigators to conclude that daily CXRs in the SICU should only be obtained when clinical necessity is suspected.14 15 Graat and colleagues demonstrated that only 5.8% of routine daily CXRs in a mixed medical and surgical ICU revealed unexpected findings, and only 2.2% led to a change in management.13 16 Others have since demonstrated a low diagnostic and therapeutic utility of routine daily CXRs in the ICU,4 and that a strategy in which CXRs are obtained only on clinical indication, when compared with routine practice, had no undesirable impact on hospital LOS or mortality.5 6 Since the completion of these reports, two meta-analyses have corroborated the safety of a more restrictive, indication-directed chest imaging strategy.7 8 In response, the ACR has since modified their position, now stating that routine daily CXRs in the ICU are unneeded and should only be obtained for specific clinical indications.12 However, this strategy has yet to be uniformly adopted.

Given current evidence, rather than set out to firmly establish the safety of an indication-restricted CXR strategy, we sought to change our local practice to one that would provide greater efficiency and value to our ICU practice. The findings in this study support our primary hypothesis that staff education and elimination of the ‘daily’ order option can reduce the number of potentially unnecessary CXRs, and hence increase the percentage of high-value CXRs. More specifically, we eliminated between 1089 and 1274 potentially unnecessary CXRs annually in our combined ICUs, equating to an estimated annual savings of around $200 000. Using CXR cost estimates published elsewhere,11 the savings could be higher than this and do not account for further reductions in downstream testing costs precipitated by false-positive CXR findings. Because we chose to normalise monthly CXR volumes to a ‘per 1000 IVDs’ ratio, and the number of IVDs actually trended down over the study period in both ICUs (figures 1 and 2), this in turn actually diminished our ultimate savings from avoided CXRs. Had the number of IVDs not decreased over the study, the savings would have been even greater.

Because CXRs were ordered only on the basis of a clinician-identified indication following our intervention, we feel it reasonable to assume that the resulting reduction in automated daily CXRs led to an increase in clinical value to those CXRs obtained following the intervention. Although we did not specifically set out to establish the safety of a more restrictive, indication-directed ICU chest imaging strategy, we were able to establish that there was no increase in the number of event reports related to missed endotracheal tube or central catheter malposition. Furthermore, there was no recognisable reduction in the overall quality of ICU care, at least as measured by ICU LOS and O:E mortality.

The absence of any event reports regarding unrecognised tube malposition likely points out a weakness in this secondary outcome and suggests an overall underutilisation of our event report system for these safety concerns. However, 39 reports of self/unplanned extubation were uncovered during the study period. When these events were normalised to IMV census days, rates actually declined from 6.4 to 4.2 unplanned extubations per 1000 IVDs in the MICU and from 2.3 to 1.1 per 1000 IVDs in the SICU. Thus, provided that there was no change in reporting habits, it does not appear that our change in practice led to any increase in unplanned dislodgements of endotracheal tubes.

In our efforts to change the local culture of practice in our hospital, we borrowed from the model developed at the Johns Hopkins Armstrong Institute of Patient Safety to promote the translation of evidence into practice: engage, educate, execute and evaluate.17 We began with engaging and educating our staff in both ICUs on the documented low clinical yield of automated daily CXRs. We then heavily borrowed from this model to execute our change in practice by redesigning the care process through standardisation, checklist-based prompts and developing a ‘hard stop’ in our ordering systems to eliminate the option for automated daily CXRs. Our system-based practice change also required a built-in prompt to avoid missing the opportunity for chest imaging when indicated, but this had to be simplified to facilitate its success. For example, in their efforts to reduce venous catheter-related bloodstream infections, by simply prompting the clinician to ask on a daily basis whether a catheter was still needed, Berenholtz and colleagues avoided the pitfalls of overcomplexity that arise from trying to develop strict inclusion and exclusion criteria that bypass clinical decision making.18 19 We likewise chose to avoid any complex algorithms that dictated when a CXR was or was not indicated, and simply prompted clinicians daily to order a CXR when circumstances were felt to warrant one. This, alone, led to a 64% reduction in ordering habits in the MICU. We believe this also helped to avoid the potential for underutilisation of clinically warranted CXRs, as an ‘out of sight-out of mind’ consequence of eliminating the ‘daily’ CXR option. Although we cannot prove that this daily prompt resulted in a lower number of missed conditions, our data suggest that this practice did not coincide with any increase in undesirable outcomes.

We also sought to reinforce the gradual adoption of practice change through regular monitoring (evaluation) of our primary outcome variable (CXRs/1000 IVDs). One might expect that by simply eliminating the ‘daily’ CXR option, we should have seen an almost immediate drop in CXRs. Instead, this process evolved over a 1-year period of time. This was likely in part due to the staggered roll out of our interventions, with changes in our EHR-based orders and daily checklist occurring months after beginning efforts to engage and educate staff (figures 1 and 2). However, our team also anecdotally noted that this process of practice change required championing on the units, with frequent re-education of staff. At first, when radiology staff arrived on the units in the morning without orders for CXRs on every patient on IMV, they would often prompt residents to place an order for what was interpreted as an error of omission. This habitual ‘local culture’ practice took time to extinguish through reminders and re-education. As success in reducing our rate of CXRs on non-admission, non-procedural census days was fed back to staff, these positive outcomes helped reinforce our change in practice.

We should also acknowledge that a quality initiative will likely achieve more universal buy-in from staff when the effort seeks to eliminate waste in resources and time rather than building on the already high burdens of work and documentation imposed on hospital staff. Experts assert that working ‘smarter, not harder’ is the better approach to improving quality in healthcare,20 pointing out that in industry, the ‘work harder’ approach can often bridge performance gaps in the short run, but are often not sustainable.21 For instance, best practice bundled care for prevention of ventilator-induced pneumonia has been estimated to add a median of 115 min per day onto the workload of nurses,22 perhaps helping to explain suboptimal compliance rates even within grant-funded research projects.23 In the current project, the low time investiture required from staff, and regular feedback on resources saved likely had a reinforcing effect on the outcome of our initiative.

Conclusions

The impact of this study is not based on re-establishing the low diagnostic yield of daily automated CXR, but rather on using methods of organisational practice change, some established and others novel, to transition local ICU culture to one that lends higher value to chest imaging practice in the ICU. We thus conclude that ICU practice of automated daily CXRs can be effectively transitioned to a higher value strategy of indication-based chest imaging through engaging and educating staff, developing a hard stop within the EHR to eliminate the ‘daily’ order option, and adding a simplified, dichotomous prompt to the ICU daily checklist (CXR ‘indicated’ or ‘not indicated’) to avoid missed opportunities for imaging when clinically warranted. Furthermore, these initiatives must be championed through regular feedback and reminders to staff, in order to slowly erode the culture of daily chest imaging in the ICU. Through this practice, comparable-sized hospitals can expect to save hundreds of thousands of dollars annually without any compromise to patient safety, ICU mortality or LOS, and in the end, impart greater value to the fewer CXRs obtained in routine ICU practice.