Plan-Do-Study-Act Methodology: Refining an Inpatient Pediatric Sepsis Screening Process.

Pediatric sepsis remains a leading cause of death of children in the United States. Timely recognition and treatment are critical to prevent the onset of severe sepsis and septic shock. Electronic screening tools aid providers in identifying patients at risk for sepsis. Our overall project goal was to decrease the number of sepsis-related emergent transfers to the pediatric intensive care unit by optimizing sepsis screening tools, interruptive alerts, and a new paper tool and huddle process using Plan-Do-Study-Act (PDSA) methodology.
METHODS: Our team utilized historical data to develop inpatient electronic sepsis screening tools to identify pediatric patients at risk for sepsis. Using PDSA iterative cycles over 3 months, we tested the design of an interruptive alert, paper tool, and a new sepsis huddle process.
RESULTS: During the PDSA, the clinical teams conducted huddles on all patients who received an interruptive alert (n = 35). Eighty percent of huddles had a 5.7 minute average response time and an average duration of 5.3 minutes. Completion of the huddle outcome notes occurred 83% of the time, and 70% had feedback related to the alert, paper form, and huddle process. The number of days between sepsis-related emergent transfers to the pediatric intensive care unit increased from a median of 17.5 to 57.5 days, with a single point as high as 195 days between events.
CONCLUSIONS: The inpatient sepsis team learned valuable lessons using PDSA methodology. The results of the iterative cycles allowed the team to optimize and refine the tests of change. System-wide implementation benefited from the application of this quality improvement tool.

INTRODUCTION

Pediatric sepsis is a severe infection that can quickly become life-threatening and remains a leading cause of death in children in the United States.[1,2] In the United States alone, more than 75,000 children develop severe sepsis each year—roughly 200 children per day.[3] From 1995 to 2005, there was an increased incidence of pediatric sepsis, with a declining mortality rate from 10.3% to 8.9%.[4] Neonates, infants, and children with chronic medical conditions comprise a large percentage of those with morbidity and mortality from severe sepsis or septic shock.[4-6] Among children who develop sepsis worldwide, 49% have a comorbid condition that leaves them vulnerable to infection. The most common comorbidities in infants involve chronic lung or congenital heart disease. Children ages 1−9 have an underlying neuromuscular disease, and adolescents have preexisting cancers.[7] Studies suggest that approximately 6,800 children will die from sepsis annually, which is more than pediatric deaths related to pediatric cancers. The treatment of severe pediatric sepsis also carries a sizeable monetary burden with an estimated cost of $5 billion in the U.S. annual healthcare expenditures.[4,6] In a recent article, national pediatric sepsis experts are working together to define a preliminary pediatric sepsis event surveillance definition to estimate the national burden, outcomes, and trends of pediatric sepsis.[8]

Despite its rising prevalence, sepsis is treatable. Timely recognition and management of sepsis are critical to prevent the onset of severe sepsis and septic shock.[9-11] Unfortunately, the rapid diagnosis of pediatric sepsis is often difficult in hospitalized patients. Age-specific vital signs and development-specific clinical parameters further complicate early detection. Studies show that delays in recognition cause delays in treatment, including the delivery of antibiotics, fluids, and supportive care.[12]

Ninety-six percent of all nonfederal acute care hospitals have an electronic health record (EHR) certified by the Department of Health and Human Services.[13] These information systems contain comprehensive patient data stored during a patient’s stay in the emergency department (ED) or hospital. Potential uses of these data include risk stratification and early prediction systems. These systems are used to notify and promote early intervention in various clinical situations, including pediatric sepsis. Studies show that automated electronic sepsis screening tools in pediatric EDs and inpatient settings demonstrate earlier identification of patients at risk for sepsis.[14-17]

Our institution is a large, urban, free-standing, quaternary, academic children’s hospital, with over 1.4 million patient visits a year. In 2013, the hospital created a multidisciplinary sepsis team to focus on timely recognition and goal-directed therapy in the ED. A few years later, in 2016, our organization joined forces with other children’s hospitals across the nation as a member of Improving Pediatric Sepsis Outcomes. This Children’s Hospital Association national collaborative aims to reduce sepsis-attributable mortality and improve survivor outcomes through early identification and treatment. The following year, the ED deployed an automated, electronic sepsis risk screening tool.

In 2018, the focus of the hospital sepsis team shifted to inpatient services. One of their tasks was to identify patients at risk for sepsis and provide early intervention. To accomplish this, the inpatient team developed a new huddle process, a bedside paper form, and electronic sepsis screening tools. When deployed, these tools would alert nurses to patients at risk for sepsis. Providers were expected to conduct a huddle within 10 minutes of notification and document the sepsis huddle outcome in the patient’s EHR. This new sepsis process is similar to “watcher huddles” implemented at a large tertiary children’s hospital free-standing academic center with a significant reduction in transfers associated with intubation, vasopressors, or significant fluid resuscitation within 1 hour before or after intensive care unit (ICU) arrival.[18]

To study how providers interact with the electronic screening tools, notifications, and a new paper tool and huddle process, we opted to use a Plan-Do-Study-Act (PDSA) model for healthcare improvement.[19,20] The results of the PDSA iterative cycles would be used to refine the screening tool, notification design, and the sepsis huddle process before spreading across the organization.

Methods

Model Development

As preliminary work for the PDSA, we utilized 15 months (March 2016 to June 2017) of inpatient data to develop electronic screening tools to identify patients at risk for sepsis. The initial dataset included 367 unexpected transfer patients to the pediatric intensive care unit (PICU). Before transfer, these patients were not in the ED, procedure centers, interventional radiology, or the operating room. The unexpected transfer population also included patients who required a rapid response or code blue team or became an emergent transfer to the PICU. Using a definition from Ohio Children’s Hospitals’ Solutions for Patient Safety, an emergent transfer is a patient transferred from an acute care floor to the PICU who is intubated, placed on inotropes, receives chest compressions, or is given ≥3 fluid boluses in the 60 minutes before or after transfer to the PICU.[21]

An intensive care physician reviewed the unexpected and emergent transfer data using Goldstein criteria[22] to determine those patients who were transferred related to severe sepsis or septic shock. The resulting dataset of 65 patients included some children on the cardiology service and patients with known malignancy. At baseline, these patient populations typically have abnormal features consistent with their underlying diagnoses (eg, poor perfusion and/or low leukocyte counts). They would not necessarily indicate that the patient was at risk for sepsis. For this reason, we opted to exclude these patient populations. The final dataset included 51 emergent or unexpected transfers to the PICU with severe sepsis or septic shock.

As part of the electronic screening tool design, we abstracted data from our EHR via the Clarity database (Epic Systems, Corp., Electronic Health Record company). We selected 22 potential or candidate features (Table 1) of the dataset using elements and parameters from the International Pediatric Consensus Conference: Definitions for sepsis and organ dysfunction in Pediatrics, the American Academy of Pediatrics Pediatric Sepsis Collaborative, and the pediatric deterioration and Pediatric Early Warning Score literature.[22-25] We used QlikView, a business intelligence software provided by Qlik (Version 11.2 SR12; Release date June 9, 2015; Radnor, Pa.), for variable optimization on a random selection of patients in a train and test cohort. We created 2 sepsis screening models with different criteria and weighted scores, thresholds for alerting, sensitivities, and specificities during this development phase. After completion of the sensitivity analysis, we selected model thresholds through careful analysis of the trade-off between the number of false alerts triggered and the proportion of septic patients identified.

Table 1.

Candidate Features

The first model optimized sensitivity while minimizing the number of false-positive alerts. This “more sensitive” configuration of the screening tool used 15 assessment features and parameters with different weighted scores and a defined threshold to alert (Table 2). This model had a sensitivity of 87% and a specificity of 86.2%. It would trigger an interruptive alert to providers, on average, 6 times per day (average daily census was 176.7 during the historical period) on nonintensive care medical and surgical units and patients without known malignancy.

Table 2.

Sensitive Model Assessment Criteria

After we configured the sensitive screening model, we focused our effort on creating a more specific screening model. The final assessment features and parameters (Table 3) for this particular model included 3 of the candidate biomarkers—procalcitonin, C-reactive protein, and lactate—and different features and parameters that combined heart rate and respiratory rate. Because sepsis is a dysregulated host immune response to infection, we were not surprised that this model included some or all of the candidate serum biomarkers. This “more specific” model had a sensitivity of 78.3% and a specificity of 93.7%. The QlikView dashboard predicted this model would trigger an alert, on average, 2.5 times per day on nonintensive care medical and surgical units and patients without known malignancy.

Table 3.

Specific Model Assessment Criteria

Once we finished the 2 screening models’ configuration, we sorted the number of alerts identified by each of the electronic screening processes by inpatient service and unit. Excluding the intensive care areas, the hospital pediatrics, and infectious disease services had the highest predicted sepsis alerts. These 2 units also had the highest number of emergent transfers related to sepsis in the historical cohort. These services would become the focus of the PDSA iterative cycles. Institutional review board approval was not required as this work constituted quality improvement activities and not human subjects research.

Plan-Do-Study-Act

Plan

Our initial test of change was to study how healthcare providers, specifically nurses and other providers, interact with the screening tool, paper form, and huddle process. To notify providers a patient was at risk for sepsis, we built an interruptive alert into the EHR. Once a patient met the threshold, the alert would appear to the bedside nurse. The resident would be paged to perform a bedside huddle using a new paper tool (Fig. 1). The paper tool would help us identify how providers interacted with the alert, aid the provider in determining the huddle outcome, and allow written feedback to optimize the design of the alert, paper tool, and huddle process. Before the PDSA, education was provided to providers and nursing staff.

Fig. 1.

Paper sepsis risk huddle form. ACT, Assessment and Consultation Team; BP, blood pressure; GCS, Glasgow Coma Scale; HR, heart rate; I&O, input & output; LOC, level of consciousness; PEWS, pediatric early warning score; RR, respiratory rate; SOD, safety officer of the day; SpO2, blood oxygen saturation level; UOP, urinary output.

Do

Using the paper tool to review vital signs, assessments, laboratory values, and signs of organ dysfunction, members of the huddle were required to determine 1 of 3 potential huddle outcomes: continue routine care, initiate a watchstander process (assessment and mitigation plan), or call the Assessment and Consultation Team. The ACT at our institution is similar in function to a rapid response team. In addition to huddle outcomes, huddle team members recorded response times and huddle durations on the paper tool. Following the huddle, the resident utilized a sepsis order set and wrote a sepsis huddle outcome note using an EHR note template created for this quality improvement project.

Study

Sepsis project team members collected the paper tools and reviewed all written feedback about the new sepsis processes. The team reviewed the huddle outcome notes for content, assessment, and plans during the PDSA study cycles.

Act

In support of our safety culture, the informatics team optimized the nursing alert (Fig. 2) to include “Stop and Resolve” text. Also, we identified providers questioning the validity of the screening tool to identify patients at risk for sepsis correctly. Because the alert did not provide the specific sepsis features and parameters met by the patient, providers lacked confidence in the electronic screening tools. Consequently, the informatics teams visually optimized the alert by displaying the abnormal findings concerning for sepsis in the alert text.

Fig. 2.

Optimized nursing interruptive alert. RN, Registered Nurse; WBC, white blood count.

After a review of feedback and written notes on the paper tool, we enhanced the clinical descriptions of the huddle outcomes and embedded them in a real-time EHR clinical reference guide. This guide provides parameters for each huddle outcome in a checklist format and includes considerations for abnormal vital signs, laboratory values, assessments, and signs of end-organ dysfunction.

After analyzing the screening models’ performance, we identified the sensitive model lacking specificity for patients with complex medical conditions. This deficiency resulted in many false-positive alerts secondary to abnormal vital signs and physical examination findings present at baseline in this population of patients. Thus, the specific model was only utilized to screen patients with diagnoses associated with static encephalopathy. This revision improved performance in this complex, high-risk patient population.

The hospital-wide screening tools excluded patients admitted to cardiology, psychiatry, ICUs, and patients with known malignancies. We utilized our marketing team to brand educational resources and EHR tools. Nursing staff completed education using the Children’s Hospital Association Sepsis Modules,[26] and residents attended a sepsis lecture to receive education for the PDSA. Providers and nursing staff also attended simulation sessions.

RESULTS

The 3-month study period helped the sepsis team learn how providers interacted with the interruptive alert, the paper tool, and the new huddle process. During the PDSA, the clinical teams conducted huddles on all patients who received an interruptive alert (n = 35). Eighty percent of huddles had a resident response time below 10 minutes, with an average of 5.7 minutes. The average duration of huddles was 5.3 minutes. Of the completed huddles, 83% had documentation of a huddle outcome note, and 70% had feedback related to the new huddle process, alert, and paper tool.

To assess clinical outcome measures over time, we utilized a g-chart control chart (Fig. 3). The process interventions are annotated on the g-chart at the time interventions occurred. The number of days between sepsis-related emergent transfers to the PICU increased from a median of 17.5 to 57.5 days with a single point as high as 195 days. The most notable improvement occurred between October 2018 and April 2019.

Fig. 3.

Calendar days between emergent transfers attributed to sepsis. **Control limit for each baseline level represents a constant number of patient days but varies because it is expressed in terms of calendar days. ET, emergent transfer.

DISCUSSION

Pediatric sepsis is a serious infection that can become life-threatening and is a leading cause of death in children in the United States. Neonates, infants, and children with chronic medical conditions comprise a large percentage of those with morbidity and mortality from severe sepsis or septic shock. Emerging literature describes that the outcomes and clinical characteristics of pediatric patients who transferred to the ICU within 24 hours of admission[27-29] and improved EHR detection systems, along with bedside shock huddles, may prevent unplanned emergent transfers to the ICU.[30] Our overall project goal was to decrease the number of emergent transfers to the PICU related to sepsis. We selected the PDSA methodology to test a set of process changes as part of the larger inpatient project. We used the results of the 3-month PDSA to learn how providers interacted with an interruptive alert, a paper tool, and a new huddle process. The iterative cycles produced ideas, comments, and suggestions and allowed us to optimize the electronic design, huddle process, and clinical reference tools before organizational spread.

An essential component of quality improvement is monitoring the performance of process interventions on control charts. In the case of rare events, a g-chart can assess the stability of process changes over time. Because rare events occur at very low rates, traditional control charts, like the p-chart, are typically not effective at detecting changes in event rates. Even though there are reports[4,5,31,32] of a rising prevalence of sepsis in pediatric patients, overall disease prevalence is rare, making the use of a g-chart the best means to monitor our process improvement over time.

We isolated our initial small scale tests of change to 2 clinical services in March 2018. In May 2018, we implemented the revisions to the processes. After the PDSA cycles, the sepsis team deployed the revised sepsis screening algorithm, visually optimized alert, refined huddle process, and clinical reference guide across our hospital in September 2018. The g-chart is annotated with the beginning of the PDSA cycles and subsequent hospital-wide implementation to demonstrate the process interventions. Remarkably, the median number of days between events increased from 17.5 to 57.5 and had an individual point as high as 195 days between emergent transfers to the PICU related to sepsis. This significant improvement, to a single point as high as 195 days between events, occurred after hospital widespread of the new inpatient sepsis tools and huddle process. Although the median days between events had improvement before the implementation of all inpatient process interventions, the most significant improvement occurred after organizational implementation.

Our single set of process changes occurred within a larger project to improve the timely recognition and treatment of admitted patients with sepsis. The inpatient sepsis team also leveraged resident education and simulation sessions to improve recognition of the septic patient. These interventions occurred just before our hospital-wide deployment of the PDSA process changes. Although the resident education and simulation interventions alone may have contributed to improved outcomes, these interventions likely had a cumulative effect on the significant increase in days between emergent transfers. Also, our inpatient team cannot fully deduce any single process intervention as providing the most clinical impact. However, after careful review of the improved emergent transfer rate, we propose that the PDSA process interventions likely contributed the most to the significant clinical improvement over time.

In the same aspect, we do not fully understand the impact of the ED screening tool (presumed earlier treatment of patients at risk for sepsis) before implementing inpatient electronic screening tools and huddle process. However, it cannot be understated that the most significant increase in the days between sepsis-attributable emergent transfers to the PICU occurred after the implementation of the inpatient sepsis screen tools and process changes. One could argue that an emergent transfer to the PICU from a medical or surgical floor is an extremely ill pediatric patient. To be an emergent transfer, these patients require multiple fluid boluses, pressors, and/or intubation. Reasonably, it seems less likely that earlier identification of patients at risk for sepsis in the ED would require emergent transfer to the PICU.

Variables such as hospital census, seasonality, and annual provider competency requirements are not adequately addressed within our limited study period. However, our teams pursue these variable inclusions with the continued evaluation of clinical outcome measures. Future efforts directed by our hospital-wide sepsis steering team and environment-specific teams, such as in the ED and inpatient areas, seek to optimize the performance of the electronic screening tools, decision support, and efficiency of provider workflows. As such, evaluation of the impact on the before mentioned variables may alter the design and functionality requirements of the currently implemented tools and process.

CONCLUDING SUMMARY

The inpatient sepsis team learned valuable lessons using PDSA methodology. The results of the iterative cycles allowed the team to optimize and refine the tests of change. System-wide implementation benefitted from the application of this quality improvement tool.

DISCLOSURE

The authors have no financial interest to declare in relation to the content of this article.

ACKNOWLEDGMENTS

The authors would like to thank Donnie Clark for statistical analysis and model development; Charlie Macias, Service Line Quality Improvement Coordinator, Quality Improvement Services, and Tricia Montgomery, MPH, Service Line Coordinator, Quality Improvement Services for their leadership on this project.