Trainee-led Engagement of the Care Team Improves Application of an Institutional Blood Culture Clinical Decision Algorithm to Pediatric Oncology Inpatients: A Single-institution Quality Improvement Project.

Meaningful engagement in quality improvement (QI) projects by trainees is often challenging. A fellow-led QI project aimed to improve adherence to a blood culture clinical decision algorithm and reduce unnecessary cultures in pediatric oncology inpatients.
Methods: We visualized preintervention rates of blood cultures drawn on pediatric oncology inpatients using a control chart. Following the introduction of the algorithm to our division, an Ishikawa fishbone diagram of cause-and-effect identified two areas for improvement: prescriber education on the algorithm and targeted feedback on its use. We developed two interventions to support algorithm awareness and use: (1) bundled educational interventions and (2) targeted chart review and feedback. Fellows reviewed >750 blood culture episodes and adjudicated each as "adherent" or "nonadherent" to the algorithm. In addition, fellows provided direct feedback to prescribers regarding nonadherent episodes and discussed strategies for algorithm adherence.
Results: Blood culture rates in preintervention, intervention, and follow-up periods were 33.35, 25.24, and 22.67 cultures/100 patient-days, respectively. The proportion of nonadherent culture episodes decreased from 47.14% to 11.11%. The use of the algorithm did not prolong the time to cultures drawn on patients with new fever. Seventy-five percent of fellows provided feedback to inpatient teams on algorithm use. Following this project, trainees reported feeling more qualified to apply QI principles to patient care. Conclusions: Implementation of a clinical decision algorithm reduced the rate of cultures drawn on pediatric oncology inpatients. Fellow-led education of the care team decreased the proportion of nonadherent culture episodes and provided active engagement in QI.

INTRODUCTION

Practice-based learning, which includes quality improvement (QI) initiatives, is a core competency of the Accreditation Council of Graduate Medical Education (ACGME). Additionally, Health Care Quality is one of six areas of the ACGME’s Clinical Learning Environment Review (CLER),[1] requiring trainee physicians to be “actively involved in QI projects in interprofessional teams, focused on resource use, aligned and integrated with the site’s priorities; and with active oversight by QI leadership.” However, a comprehensive CLER review in 2016 found that trainees were often only exposed to QI through didactics rather than experiential learning.[2] Obstacles to meaningful trainee engagement in QI include competing demands, lack of involvement by faculty experts, lack of project ownership, and lack of trainee knowledge of QI. Although examples of trainee QI scholarship exist in the literature,[3] it is generally rare for trainee involvement in QI projects to produce scholarship, as evidenced by a recent meta-analysis yielding only 28 publications from trainee QI projects.[4]

Our training program sought to overcome these barriers through a fellow-led QI project. In 2014, our institution first introduced a clinical decision support tool to guide prescribers on blood culture use and reduce unnecessary cultures.[5,6] In the Pediatric Intensive Care Unit (PICU) population, this intervention decreased the rate of cultures drawn by 46% and found no change in episodes of septic shock.[5] The PICU population included pediatric oncology and hematopoietic stem cell transplant (HSCT) patients requiring ICU-level care. However, it was unclear how prescribers used the clinical decision algorithm in care for patients admitted to the general pediatric oncology service. Therefore, the pediatric oncology fellows decided to improve blood culture utilization on the inpatient service.

Clinicians have a low threshold to obtain blood cultures in oncology patients, given their immunocompromised status. Yet, these tests may be low-yield or result in false positives when obtained only from central venous catheters (CVCs).[7,8] Although guidelines exist around culture practices for neutropenic oncology patients presenting with first fever,[9] there is wider variation regarding culture practices in pediatric oncology/HSCT patients in other circumstances, such as persistent fever.[10-12] Our fellows noted that applying the institutional algorithm was confusing and, at times, led to perceived misuse, thus motivating the selection of this topic for their QI project.

To improve the implementation of the clinical decision algorithm on the pediatric oncology inpatient service and decrease unnecessary blood culture use, the fellows collaborated with faculty leaders and experts from infectious diseases (ID) and implementation science. The primary aim was to reduce unnecessary culture episodes to ≤10% over 2 years.

METHODS

Context

Johns Hopkins Hospital is a metropolitan tertiary care center in the Mid-Atlantic region of the United States. The Division of Pediatric Oncology provides oncology and HSCT care for patients younger than 25 years old, with a patient population that is approximately 50% White, 24% Black, 6% Asian, 5% Hispanic, and 15% other/unreported. The division sees ~200 new patients annually. Inpatients are primarily admitted to a dedicated 21-bed unit. Three teams provide inpatient care: two pediatric oncology/HSCT attending physicians, two pediatric oncology fellows, two pediatric residents, and two advanced practice providers (APPs). Many clinicians rotate through these teams (20 attending physicians, 6 first-year pediatric oncology fellows, approximately 25 pediatric residents, and 8 APPs). Each patient is also cared for by one pediatric oncology nurse per shift drawn from a pool of over fifty nurses. Thus, an essential component of this project was effectively disseminating care guidelines and sharing feedback across all care providers.

Background

In April 2014, the PICU instituted a blood culture clinical decision algorithm.[5] Oncology physicians participated in discussions of its design; however, awareness and use remained inconsistent on the inpatient oncology service. In October 2016, an updated algorithm was reintroduced at a divisional faculty meeting (Fig. 1). The 20 pediatric hematology-oncology fellows identified improving blood culture practices as the priority for QI, with support from faculty leadership in oncology and ID.

Fig. 1.

Hospital-wide blood culture decision algorithm as of September 2016.

This report is of a single-institution QI project of blood culture practices for pediatric oncology patients admitted to the general floor. Therefore, we excluded blood cultures obtained in the outpatient oncology clinic, the pediatric emergency department, and the PICU. These cultures involved separate prescriber teams.

Interventions

Project Leadership and Team

Leaders and collaborators on this project began meeting in August 2016. Several team members developed the clinical decision algorithm as part of the prior institutional work.[5] Project leaders included two HSCT faculty members, the pediatric hematology-oncology fellowship director, and two senior fellows (K.M.L. and D.J.Y.) serving as QI committee chairs. Key additional collaborators included leadership from pediatric ID and nursing colleagues. The team met regularly over the Intervention period to review data collected and analyze results (Fig. 1, Supplemental Digital Content 1, which describes timeline of the project includes preintervention, intervention (divided into planning, educational interventions, assessment/feedback interventions), and follow-up. In addition, specific activities are noted, including clinical team information sessions and feedback (c); educational presentations within the fellowship, division, and local institution (p); QI team meetings (q); and data collection/analysis efforts (d), http://links.lww.com/PQ9/A367).

Define-measure-analyze-improve-control QI Process

This QI project employed the LeanSigma Define-measure-analyze-improve-control (DMAIC) framework (Fig. 2, Supplemental Digital Content 1, which describes diagram of DMAIC QI process, http://links.lww.com/PQ9/A367), identifying unnecessary blood cultures in pediatric oncology inpatients as a process requiring further improvement. The start of the Intervention period was considered the introduction of the algorithm in October 2016 and was supported by additional educational and feedback interventions. In February 2017, relevant data regarding blood culture utilization was shared with key QI stakeholders. Ishikawa cause-and-effect diagramming identified factors contributing to unnecessary blood culture use in this population (Fig. 2). Fellowship trainee clinicians identified a lack of awareness of the algorithm and lack of feedback on its use as two other priorities for intervention.

Fig. 2.

Ishikawa cause-and-effect diagram to analyze potential causes of unnecessary blood culture draws. Circled areas represent targets for the QI project.

Bundled Educational Interventions

Targeted educational interventions to increase awareness of the algorithm began in July 2017 (Fig. 1, Supplemental Digital Content 1, http://links.lww.com/PQ9/A367). We held several didactic presentations and multidisciplinary discussions for faculty and fellows between July and October 2017. New fellows and residents received focused education on the algorithm during each group’s orientation to the unit. Additionally, the algorithm was widely posted in clinical work areas on our unit.

Intervention to Provide Feedback on Algorithm Use

Fellows conducted a chart review of sampled blood culture episodes (see Measures for details). First, they identified whether episodes adhered or did not adhere to the algorithm. Then, upon identifying nonadherent episodes, fellows reviewed chart notes around the episode to identify additional contributing factors. Finally, fellows discussed the scenario with the prescriber team, who had been on service at the time of the episode, to build awareness of the algorithm and understand gaps in knowledge regarding its application.

Measures

Number of Blood Cultures Drawn per 100 Patient-days

We calculated the monthly rate of cultures collected per 100 patient-days. Blood cultures were extracted monthly from the inpatient electronic health record (EHR; January 2015–June 2016 Allscripts Sunrise; July 2016–December 2018 EPIC), and the hospital provided census data.

Nonadherent Culture Episodes

To evaluate adherence to the clinical decision algorithm, fellows reviewed a sampling of blood cultures drawn on pediatric oncology inpatients from July 2016 to February 2018. Because the site and number of cultures recommended by the algorithm vary based on indication for culture, we defined a blood culture “episode” as the total number and sites of cultures required for a single culture indication. For example, in a patient with a new fever (>48 hours from last fever), a blood culture “episode” would include culture from all CVC lumens and a peripheral culture. Fellows followed a chart review protocol and completed a data collection tool to adjudicate whether culture episodes adhered to or deviated from the algorithm. The data collection tool also asked fellows to annotate why nonadherent culture episodes deviated from the algorithm to inform future interventions.

Analysis

We performed both quantitative and qualitative analyses to investigate the effect of interventions on blood culture practices in our division. These included the following with relevant methodology detailed below each analytic.

Quantitative Metrics

We calculated the monthly rate of cultures collected per 100 patient-days and the average rate of blood cultures during three time periods: preintervention (January 2015–October 2016); intervention (November 2016–June 2018); and follow-up (July 2018–December 2018). A control chart was generated in Microsoft Excel using an institutional tool based on standard methodology. This control chart permitted us to visualize the changes in the rate of blood culture collection with time. Data input included: total cultures in a calendar month, total patient-days in a calendar month, and investigator-specified periods for calculating baselines and control limits (defined as the preintervention, intervention, and follow-up periods). The control chart calculates the average rate from the data and upper and lower control limits accounting for changes in the sample size. Control limits at 3 SDs from the mean were determined by the data. An incident rate ratio compared the average rate of cultures per 100 patient-days in the preintervention and follow-up periods.

The fellow-led chart review calculated the proportion of nonadherent culture episodes each year (2016–2018) as the ratio of the number of culture episodes adjudicated as nonadherent to the algorithm compared to the total adjudicated culture episodes. Due to a transition in the hospital EHR in July 2016, we conducted the chart review only on inpatients admitted beginning in July 2016. Therefore, cultures were sampled in 2-week blocks from July 2016 to February 2018, with each fellow assigned to review culture episodes from one or more blocks.

To address the concern that consulting the clinical decision algorithm might delay patient care, we examined the time-to-culture draw and time-to-antimicrobial change from new fever detection on a subset of inpatients. “New fever” was defined as >48 hours from a prior fever. Episodes from August to November 2016 were compared to August to November 2017.

Qualitative Metrics

Fellows and faculty members provided narrative comments regarding perceived barriers in using the algorithm (Table 1, Part I). We also gathered comments from the fellow-faculty dialogue regarding cultures adjudicated as nonadherent to inform potential nuances or detailed situations where the algorithm did not fit a particular clinical situation (Table 1, Part II). Finally, using a survey, we assessed the effect of participation in this QI project on fellows’ perceived ability to apply QI principles to patient care and compared responses to historical surveys of our fellows.

Table 1.

Qualitative Metrics

Part I: Feedback on Perceived Barriers to Using Clinical Decision Algorithm by Pediatric Oncology Physicians
Perceived Barrier	Example Comments
Subspecialty culture	“What is the definition of immunocompromised status for the decision tree? I know it mentions ‘severely immunocompromised’ but if it’s not severe are we not consider[ing] patients non-immunocompromised (newly diagnosed ALL, or patient on chemo without neutropenia?)”
Practice standards	“If I am reading this algorithm correctly, it suggests that if patients are stable and still febrile, they should only be cultured once every 48hrs. I believe our standard is once every 24 hours if still febrile.”
Lack of recognition of problem	“…the common practice is that these follow up cultures for fever are … drawn from the central line.”
Process clarity	“Practically though, I see why the additional cultures [are] drawn since … labs are done at night so would avoid entering the line again for cultures if the patient does have another fever.”
Part II: selected comments from oncology faculty on cultures adjudicated as unnecessary per the clinical decision algorithm
Reason nonadherent	Example comments from feedback
Incorrect timing	“That culture was drawn… overnight – the residents may have done it without talking to the fellow or myself. “
	“[Patient] was neutropenic with diarrhea and painful perirectal excoriations without fever. …a blood culture was obtained as a precaution due to diarrhea and broken skin integrity. Only a single culture was drawn because he had had the full set the previous day.”
Incorrect source	“For this new fever should have had peripheral and both lumens of central line cultured. We were unable to obtain peripheral, but I am not sure why the second central culture was not drawn.”
Too many cultures	“[Patient] had a history of fungal disease, so the team made the decision to be more aggressive”
	“[Patient] had prior hx of bacteremia after reporting tooth ache, so cultures were obtained again given report of tooth ache”
Peripheral not drawn/refused	“[patient’s] culture was just from the line because [primary attending] … didn’t feel that he needed to be poked for it.”
Additional sources needed	“…the culture …. should have been a full set (peripheral and central). I believe the peripheral was attempted but refused and the port was not able to be reaccessed after a needle change so it could not be cultured.”
Other	“This…fever was likely related to cytarabine”

Statistics

To compare proportions of nonadherent culture episodes across years, we constructed exact binomial confidence intervals and performed the Fisher exact test. A P value less than 0.05 was considered statistically significant.

Ethical considerations

The Johns Hopkins Institutional Review Board acknowledged this QI project. As a result, faculty who demonstrated active participation in the QI chart review and feedback process were eligible to receive credit toward their maintenance of board certification.

RESULTS

Blood Culture Collection Rates Decreased Following Interventions and Persisted in Follow-up

From January 2015 to December 2018, 7,022 total blood cultures were drawn on pediatric oncology inpatients (Fig. 3), representing 24,627 patient-days. In the preintervention period (January 2015–October 2016), 33.35 cultures per 100 patient-days were collected. In the intervention period (November 2016–June 2018), an average of 25.24 cultures per 100 patient-days were collected. In the follow-up period (July 2018–December 2018), 22.67 cultures per 100 patient-days were collected. This change represents a 32% decrease in the rate of total culture collection from the preintervention period compared to the follow-up period.

Fig. 3.

Control chart for the number of blood cultures per 100 patient-days displaying cultures from January 2015 to December 2018. Three time periods are preintervention (January 2015–October 2016), intervention (November 2016–July 2018), and follow-up (July–December 2018). Interventions are highlighted in the figure (^ = algorithm introduction, ^^ = educational intervention, ^^^ = feedback intervention).

Blood Culture Episodes Adjudicated by Trainees as Nonadherent Decreased Following Interventions

Seven hundred eighty-eight culture episodes were sampled from July 2016 to February 2018. This finding represented 1,202 total cultures of 2,639 total cultures for this period (45.55% of total cultures). The fellow-led review demonstrated that 47.14% of sampled blood culture episodes deviated from the algorithm in 2016 (Fig. 4A) before educational and feedback interventions. In 2017, the proportion of nonadherent culture episodes was 15.94%, and in 2018, the proportion of nonadherent culture episodes was 11.11%. Thus, there was a reduction in nonadherent culture episodes over the period evaluated (P < 0.001).

Fig. 4.

Results of trainee led chart review on blood cultures during QI project. A, Proportion nonadherent culture episodes identified from sampled cultures grouped by year. B, Time from new fever detected to blood cultures drawn on pediatric oncology inpatients comparing August–November 2016 and August–November 2017 time periods.

In 2016 (ie, preintervention period), the most common reason for episodes to deviate from the algorithm was that many patients had multiple CVC lumens cultured every 24 hours for persistent fever in the absence of other clinical changes suggesting bacteremia (Fig. 3, Supplemental Digital Content 1, which describes reasons for nonadherent culture episodes grouped by year, http://links.lww.com/PQ9/A367). In 2017–2018, this practice decreased from 66% of nonadherent cultures to 18% of nonadherent cultures. Qualitative feedback revealed additional reasons why culture episodes deviated from the algorithm (Table 1, Part II).

Implementation of Clinical Decision Algorithm Did Not Delay Care for Inpatients with New Fever

In a subgroup analysis comparing time-to-culture before interventions (August–November 2016) and 1 year later, the median time-to-culture on inpatients with new fever (>48 hours from last fever) was not significantly different between the groups (Fig. 4B; August–November 2016: median time = 70 minutes; IQR 30, 189 minutes; n = 32 episodes. August–November 2017: median time = 62.5 minutes; IQR 38, 99.5 minutes; n = 28 episodes). No significant difference was observed in time-to-antimicrobial change in patients with new fever (Fig. 4, Supplemental Digital Content 1, which describes time to antimicrobial change for new fever in pediatric oncology inpatients comparing August–November 2016 and August–November 2017 time periods, http://links.lww.com/PQ9/A367; August–November 2016: median time = 135 minutes; IQR 84.5, 253.5 minutes; n = 29 episodes. August–November 2017 median time = 141 minutes; IQR 97, 223 minutes; n = 24 episodes).

Participation in Division-based QI Projects Improves Trainee Understanding of Unnecessary Blood Culture Practices and Improves Quality and Safety Training

Fellowship leadership required participation in this project for trainees in our program, with all 20 fellows demonstrating active participation in this project. Fellows who identified nonadherent episodes provided feedback to the on-service attendings to inform them how the episode deviated from the algorithm and identify areas for future algorithm improvement. Seventy-five percent of fellows initiated this dynamic feedback process with outreach to faculty, and 50% of the faculty responded to engage in dialogue regarding blood culture practices. In addition, 50% of fellows surveyed in 2018 strongly agreed with the statement “I feel qualified to apply QI principles and methodology to improve patient care,” compared with 33% of fellows in 2016, before the QI project.

DISCUSSION

This report is of a single-institution, fellow-led, QI initiative on blood culture practices in the pediatric oncology inpatient population. We capitalized on previous work successfully implementing a clinical decision algorithm in PICU populations.[6] We developed interventions to improve the application of the same algorithm in the pediatric oncology inpatient population. This strategy reduced blood culture sampling in our population by 32% over approximately 1.5 years. Nonadherent culture episodes were reduced from 47% to 11% of sampled episodes. We observed a reduction in blood culture sampling beginning in November 2016, before formal targeted educational interventions. This phenomenon likely reflects several factors: (1) reintroduction of the algorithm at the October 2016 divisional faculty meeting and (2) general awareness of the QI initiative within the division, with prescribers and nursing staff becoming more attentive to practices around blood culture use. We did not observe any increase in the time from new fever detection to blood culture collection, suggesting that this clinical decision tool did not interfere with safe clinical care. Although our analysis did not examine cost, the decrease in culture rate also improves high-value health care.

An important goal of this work was to involve trainees in QI to increase the number of stakeholders and improve awareness of reducing unnecessary blood cultures. Trainees gained active experience with QI, including cause-and-effect diagramming, DMAIC project structure, data gathering, analysis, and feedback initiatives. Fellows reported feeling more prepared to incorporate QI methodology into their future work. This approach addressed many common barriers to trainee engagement in meaningful QI. The cooperative format allowed flexibility. Dividing the project into concrete tasks allowed each fellow to have a meaningful contribution.

This study has several limitations. First, it excluded pediatric oncology patients in the emergency department and PICU. Pediatric oncology prescribers would be serving as consultants, not primary prescribers, in those settings. Second, since the initiative relied on a culture result in the EHR, this analysis theoretically could have missed situations in which a blood culture was not obtained on a patient but should have been drawn. We believe this is an unlikely scenario as pediatric oncology prescribers have a low threshold to draw blood cultures. Fewer than 6% of the nonadherent sampled episodes were noted to need additional source sampling (Fig. 3, Supplemental Digital Content 1, http://links.lww.com/PQ9/A367). Finally, we could not collect data on septic shock and related ICU transfers, important outcomes that could be impacted in the event of a delay in diagnosing a bloodstream infection. Anecdotally prescribers did not observe an increase in unanticipated ICU transfers during the intervention period. Multicenter studies that can adjust for confounders such as the severity of illness and underlying conditions will be needed to monitor for rare unintended events following the introduction of diagnostic stewardship interventions.

This project utilized a manual chart review to determine the proportion of nonadherent cultures. Although this level of attention provided important details on culture practices, manual chart reviews are not often sustainable. Other institutions with initiatives to improve blood culture practices in the pediatric oncology population have modified the EHR to collect such data in real-time and provide a more standardized approach.[13] As an alternative to manual review, we found that we could examine the rate of cultures drawn per 100 patient-days in the control chart as a surrogate marker. After this project, our institution developed an automated dashboard linked to the current EHR to display a similar metric. Ongoing reminders of the blood culture algorithm, including its presence in resident and nursing workspaces, reminders to fellows at annual orientation, and similar practices in use in other areas of the hospital where pediatric oncology patients are cared for (eg, the PICU and ED) have continued to hardwire practices in to minimize unnecessary blood culture collection.

CONCLUSION

In summary, this article demonstrates the feasibility and efficacy of experiential QI education led by pediatric oncology trainees and the effectiveness of applying a hospital-wide blood culture clinical decision algorithm to the hospitalized pediatric oncology population. With multiple targeted interventions and ongoing feedback practices, we improved adherence to the algorithm and decreased total culture use without delaying patient care. This article involved all fellows in the division in active and meaningful QI, preparing trainees to adopt QI methodology in future practice. This approach will translate well to other institutions and motivate a multi-institution pediatric oncology QI initiative.

ACKNOWLEDGMENTS

We thank the 2016–2018 fellowship trainees and faculty in the Division of Pediatric Oncology at Johns Hopkins for participation in this QI project. We thank Dr. David Stockwell (Johns Hopkins) for helpful feedback.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.