Improvement of 1st-hour bundle compliance and sepsis mortality in pediatrics after the implementation of the surviving sepsis campaign guidelines.

OBJECTIVES: To study the impact of the implementation of the Pediatric Surviving Sepsis Campaign protocol on early recognition of sepsis, 1-h treatment bundle and mortality.
METHODS: Retrospective, single-center study, before and after the implementation of the sepsis protocol. OUTCOMES: sepsis recognition, compliance with the 1-h bundle (fluid resuscitation, blood culture, antibiotics), time interval to fluid resuscitation and antibiotics administration, and mortality. Patients with febrile neutropenia were excluded. The comparisons between the periods were performed using non-parametric tests and odds ratios or relative risk were calculated.
RESULTS: We studied 84 patients before and 103 after the protocol implementation. There was an increase in sepsis recognition (OR 21.5 [95% CI: 10.1-45.7]), in the compliance with the 1-h bundle as a whole (62% x 0%), and with its three components: fluid resuscitation (OR 31.1 [95% CI: 3.9-247.2]), blood culture (OR 15.9 [95% CI: 3.9-65.2]), and antibiotics (OR 35.6 [95% CI: 8.9-143.2]). Significant reduction between sepsis recognition to fluid resuscitation (152min×12min, p<0.001) and to antibiotics administration (137min×30min) also occurred. The risk of death before protocol implementation was four times greater (RR 4.1 [95% CI: 1.2-14.4]), and the absolute death risk reduction was 9%.
CONCLUSION: Even if we considered the low precision of some estimates, the lower limits of the Confidence Intervals show that the implementation of the Pediatric Surviving Sepsis Campaign guidelines alongside a qualitive assurance initiative has led to improvements in sepsis recognition, compliance with the 1-h treatment bundle, reduction in the time interval to fluid resuscitation and antibiotics, and reduction in sepsis mortality.

Introduction

Sepsis is a life-threatening organ dysfunction caused by a dysregulated host response to infection, associated with millions of deaths worldwide each year, many of them in children. In South America, its prevalence in Pediatric Intensive Care Units (PICUs) has been reported as 40% of all admissions, with almost half of the cases as septic shock, and a chance of death almost 3 times higher than in North America.

Although several consensus conferences have been held to review the diagnostic criteria and the management of sepsis since 1991,5, 6, 7, 8, 9, 10, 11 two of them specific for children and adolescents,12, 13 sepsis continues to be underdiagnosed, and compliance with the proposed protocols has been shown to be low, including in pediatric services.14, 15, 16, 17, 18, 19 As the number of pediatric studies evaluating the adherence to the Surviving Sepsis Campaign (SSC) guidelines is small, we aimed to study the impact of implementing the SSC protocol on the early recognition of sepsis and on the compliance with the recommended 1 -h treatment bundle, as well as on mortality due to sepsis in a pediatric referral hospital in Rio de Janeiro, Brazil.

Methods

Aim

The aim of this study is to verify the impact of the implementation of the Pediatric SSC protocol on early recognition of sepsis, 1 -h treatment bundle and mortality in children and adolescents.

Study design

This is a retrospective, observational cohort study that evaluated pediatric patients identified as having diagnostic criteria for sepsis during their stay in the ward or in the PICU, before and after the implementation of the SSC protocol. We followed the Standards for QUality Improvement Reporting Excellence (SQUIRE 2.0) that provide a framework for reporting new knowledge about how to improve healthcare. The study was approved by the Research Ethics Committee of the D’Or Institute for Research and Education with the number 2.719.755 (CAAE 88922518.1.0000.5249).

Setting and study period

The Rio de Janeiro State Children’s Hospital is a state referral hospital with 77 beds (10 PICU beds and 10 surgical Neonatal ICU beds) for orthopedic, surgical, oncologic, and transplant admissions (kidney and liver). It opened in March 2013, and it performs about 3800 admissions for surgeries per year, about 100 new onco-hematological patients per year, and admits an average of 250 patients in its PICU per year. The hospital does not have an open emergency department and does not perform bone marrow transplants or heart surgery. It is staffed only by specialists in pediatrics and residents, who are responsible for the general care of children in their areas and for the primary recognition of sepsis in particular. During the study period, the staff turnover rate was of 1.3% per year.

The pre-implementation period was from March 10, 2013 to September 20, 2015, when there was no regular protocol for sepsis. At that time, the board of directors and quality management decided to study the number of deaths from sepsis and what could be done to reduce it, resulting in the decision to implement the SSC protocol (from September 21 to October 18, 2015). The post-implementation period was from October 19, 2015 to April 30, 2018.

Inclusion and exclusion criteria

To identify eligible cases in the period prior to the implementation of the protocol, we have consulted the patient electronic charts searching for at least one antibiotic ordered in the prescription section indicated by the Latin American Institute of Sepsis (ILAS) or by the Hospital Infection Control Commission (HICC). With the list of eligible patients selected by this criterion, we reviewed their medical records to look for those who met the clinical and laboratory criteria for sepsis, according to the criteria of the international pediatric consensus–described in 2005 by Goldstein et al. –within the 48 h prior to the prescription of antibiotic therapy. These patients were then included in the study as part of the pre-protocol phase. We used this definition of sepsis because at the time of data collection (2013–2018) there were still no new definitions for pediatric sepsis. In addition to these criteria, all deaths whose primary or secondary cause attested in the death certificate was sepsis or septic shock were also included. We used the same criteria for the post-protocol implementation period in order to give uniformity to the methodology, although for these patients we also had data from the sepsis diagnostic forms developed specifically for the management of the protocol and completed by the team (Case Report Form, Supplementary File).

The exclusion criteria were: patients who met sepsis criteria during the washout period (September 21, 2015 to October 18, 2015); patients with chemotherapy-related neutropenia (<1.000 cells/mm3) in the 14 days prior to sepsis (as these patients followed a specific protocol from the hemato-oncology service different from the SSC protocol); and finally, patients older than one month of age but admitted to the Neonatal ICU.

Procedures for protocol implementation

The protocol implementation was preceded by the analysis of the 10 deaths from sepsis that occurred in the pre-implementation period. In none of the 10 cases the team adhered to the 1 -h bundle recommended by the SSC, and only one patient (10%) was diagnosed with sepsis by the medical team in a timely manner. The poor bundle compliance was mainly characterized by the failure to administer antibiotics, as shown by the cause and effect diagram shown in Fig. S1. As the study was retrospective, a “time zero” was defined by the research team as the moment when the patient fulfilled the sepsis criteria. Next, we considered the time that elapsed between this “time zero” and the moment in which the nurse registered the administration of the antibiotic in the medical record. If it was not administered within the first hour, this was considered a failure. Another point of concern was the appropriate administration of fluids in the first hour. In the same way that we did with antibiotics, we counted the time from the “time zero” until the administration of the fluids and considered as a failure the non-administration in the first hour or the administration of a volume below the recommended. Both non-conformities were caused by logistical problems due to centralized pharmacy services.

To solve these problems, quality improvement strategies were used with Plan-Do-Check-Act (PDCA) improvement cycles, which resulted in the following actions: a) availability of crystalloid stock in the inpatient unit; b) implementation of an assistance sheet to guide the steps of the 1 -h bundle; c) standardization of antibiotic regimens for the treatment of sepsis; d) implementation of a manual antibiotic prescription sheet to enable the immediate dispensation of the drug from the pharmacy to any member of the team, reducing the time between electronic prescription and availability of the drug in the pediatric unit; e) provision of a device for remote activation of the attending physician in cases of suspected sepsis; f) distribution of posters presenting the protocol, clarifying the diagnostic criteria and the objectives to be achieved during the first hour; g) distribution of cards for care professionals, with ranges of normal vital signs by age group and warning signs for sepsis recognition; h) training of the multidisciplinary team in simulations of patients with suspected sepsis every six months; i) definition of sepsis indicators written in the assistance sheet and disclosed on folders; j) definition of a doctor responsible for monitoring the protocol; and l) schedule of a semiannual meeting to disseminate sepsis care indicators.

Data extraction and measures of process and outcome

Demographic and clinical data were collected, including sex, age group, ethnicity, admitting specialty, site of infection, antibiotics used, clinical presentation and type of colonization. The process measures were: 1) early identification of sepsis; 2) compliance with the 1 -h treatment bundle and its three individual components (fluid resuscitation with at least 20 mL/kg of crystalloid solution, blood culture before antibiotic administration, and empiric antibiotic therapy); 3) time for fluid resuscitation; 4) time for antibiotic therapy. The outcome measure was: 5) mortality from sepsis. All these measures were evaluated before and after the implementation of the SSC protocol.

All data were extracted from patients’ electronic medical records (Tasy, Philips Healthcare), which guaranteed the accuracy of the data, especially the time intervals considered in the study.

Data analysis

The sample size was determined by the length of the pre- and post-implementation periods of the protocol, available for data collection. The consistency of the data was assessed in a random sample composed of 10% of patients. The data were tracked in detail for missing information, inconsistent values, and outliers.

Continuous variables were described as medians, and interquartile ranges and categorical variables, as proportions. The four process measures were evaluated only among patients identified as having sepsis by the medical team in the pre- and post-protocol implementation periods. Mortality was calculated as the proportion of deaths in relation to the total of patients identified by the research team as having sepsis in each period. Comparisons between the pre- and post-protocol implementation periods were performed using the Mann-Whitney test for continuous outcomes and Fisher's exact or chi-squared test for categorical outcomes. Odds ratios to each process and outcome measure and their respective 95% confidence intervals were also evaluated. A significance level of 5% was adopted, and the R software (R Core Team, Vienna, Austria) was used for all statistical analyses.

Results

A total of 187 patients were evaluated: 84 in the pre-protocol period and 103 after the protocol implementation (Fig. 1). The main characteristics of these two groups are presented in Table 1 and in Table S1. There were some small differences between the groups: patients in the post-protocol period were slightly younger and had more leukocytosis, but the other characteristics were quite similar. As for the patient's origin, in the pre-protocol phase 15 of the 84 (17.9%) were admitted to the PICU, and in the post- protocol phase 18 of the 103 (17.5%).

Figure 1

Patient selection flow chart before and after the implementation of the managed sepsis protocol.

Table 1

Demographic and main clinical characteristics of patients with sepsis.

Clinical characteristics	Pre-protocol (n = 84)		Post-protocol (n = 103)		Total (n = 187)
	n	(%)	n	(%)	n	(%)
Sex
Female	37	(44)	40	(39)	77	(41)
Male	47	(56)	63	(61)	110	(59)
Age group
Infant (1 mo-1 yr)	15	(18)	17	(17)	32	(17)
Toddler and preschool (2−5 yrs)	26	(31)	41	(40)	67	(36)
School-aged child (6−12 yrs)	12	(14)	22	(21)	34	(18)
Teens (13−18 yrs)	31	(37)	23	(22)	54	(29)
Median age in months (IQR)	75	(13−158)	44	(20−117)	53	(17−149)
Ethnicity
Asian	0	(0)	3	(3)	3	(2)
White	43	(51)	52	(51)	95	(51)
Black	6	(7)	19	(18)	25	(13)
Mulatto	35	(42)	29	(28)	64	(34)
Infection site
Abdominal	8	(9)	11	(10)	19	(10)
Genitourinary	8	(9)	11	(10)	19	(10)
Osteoarticular	4	(5)	0	(0)	4	(2)
Skin and Subcutaneous	10	(12)	18	(18)	28	(15)
Central Nervous System	3	(4)	1	(1)	4	(2)
Respiratory Tract	20	(24)	15	(15)	35	(19)
Undetermined	31	(37)	47	(46)	78	(42)
Signs and symptomsa
Fever	68	(81)	87	(85)	155	(83)
Hypothermia	11	(13)	5	(5)	16	(9)
Leukocytosis	12	(14)	30	(29)	42	(23)
Leukopenia	4	(5)	5	(5)	9	(5)
Abnormal heart rate	28	(33)	60	(58)	88	(47)
Abnormal respiratory rate	75	(89)	85	(83)	160	(86)

The total does not add up to 100% due to the occurrence of symptom overlap.

During the pre-implementation period of the protocol, sepsis was readily recognized in only 19.0% of patients (16/84 total patients), compared with 83.5% (86/103) during the post-implementation period of the SSC protocol (OR, 21.5 [95% CI, 10.1–45.7]; p < 0.001). In those patients who had a timely sepsis diagnosis in both periods (16 × 86), we observed a significant increase in compliance with the complete 1 -h treatment bundle, from 0 to 61.6% (53/86 total patients), as well as in the adherence to each of its three measures: timely initial fluid resuscitation, from 6.3% (1/16) to 67.4% (58/86), OR 31.1 ([95% CI, 3.9−247.2]; p < 0.001); blood culture before antibiotic, from 56.3% (9/16) to 95.3% (82/86), OR 15.9 ([95% CI, 3.9−65.2]; p < 0.001); and antibiotic administration in the first hour, from 31.3% (5/16 total patients) to 94.2% (81/86 total patients), OR 35.6 ([95% CI, 8.9−143.2]; p < 0.001). Fig. 2 shows the semiannual evolution of each of these items, allowing a good comparison of the two periods.

Figure 2

Semiannual evolution of the quality measurements during the study period.

Percentage of sepsis recognition (A), percentage of adherence to the first-hour treatment bundle (B) and to the three components of this bundle: C - administration of antibiotics in the first hour of the diagnosis; D - blood culture collected before the antibiotics’ administration; and E - timely initial fluid resuscitation in this first-hour period.

In addition, there was a significant reduction in the time interval between the recognition of sepsis and fluid resuscitation (152min × 12min), as well as the administration of antibiotics (137min × 30min), as shown in Table 2. A total of 10 deaths from sepsis (11.9%) were observed during the pre-protocol period, compared to 3 deaths (2.9%) during the period of use of the protocol. The risk of death from sepsis was significantly higher in the pre-protocol period compared to the post-protocol period (RR, 4.09 [95% CI 1.2–14.4]; p = 0.02), an absolute reduction in the risk of death of 9% between the two periods (Table 2).

Table 2

Comparison of outcomes between the periods before and after the implementation of the sepsis protocol.

Recognition of sepsis	Pre-protocol	Post-protocol	OR (95% CI)	p-value
Total patients with sepsis criteria, n (%)	84 (100.0)	103 (100.0)		< 0.001b
Diagnosis of sepsis by the team, n (%)	16 (19.0)	86 (83.5)	21.5 (10.12−45.66)
Compliance with the 1 -h treatment bundle (only patients with recognized sepsis)	n = 16	n = 86	OR (95% CI)	p-value
All three bundle measures, n (%)	0 (0.0)	53 (61.6)	NA	< 0.001b
Adequate fluid resuscitation,a n (%)	1 (6.3)	58 (67.4)	31.1 (3.9−247.2)	< 0.001b
Blood culture before antibiotics, n (%)	9 (56.3)	82 (95.3)	15.9 (3.9−65.2)	< 0.001c
Antibiotic in the first hour, n (%)	5 (31.3)	81 (94.2)	35.6 (8.9−143.2)	< 0.001c
Time interval (in minutes) between sepsis diagnosis and fluid resuscitation
Median (IQR)	152 (96−177)	12 (4−23)	NA	< 0.001d
Time interval (in minutes) between sepsis diagnosis and antibiotic administration
Median (IQR)	137 (39.5−188.8)	30 (17.2−45)	NA	< 0.001d
Lethality	n = 84	n = 103	RR (95% CI)	p-value
Deaths from sepsis	10 (11.9%)	3 (2.9%)	RR 4.09 (1.16−14.4)e	0.016b

OR, odds ratio; CI, confidence interval; IQR, interquartile range; NA, not applicable; RR, Relative Risk.

At least 20 mL/kg of a crystalloid solution.

Chi-squared test.

Fisher's exact test.

Mann-Whitney test.

Absolute risk reduction of death (AAR) = 9%.

Discussion

Comparing the periods before and after the implementation of the SSC guidelines as a structured protocol, associated with quality improvement interventions, we were able to demonstrate a significant increase in sepsis recognition, as well as a significant improvement in compliance with the 1 -h treatment bundle. A reduction in the time interval between the diagnosis of sepsis and fluid resuscitation–as well as the timely administration of antibiotics–was also demonstrated. Finally, we also found a significant reduction in mortality from sepsis between the two periods.

Low rates of sepsis recognition in hospitals that do not use an evidence-based protocol have been previously reported.22, 23 In this study, only 19% of pediatric patients with sepsis were recognized in a timely manner in the pre-protocol period (84% post-protocol), a finding similar to that found by Manaktala et al., who reported an increase from 21% to 71% in the correct diagnosis of sepsis before and after the implementation of a protocol. In addition to improving the diagnosis, the implementation of a sepsis bundle protocol seems to increase adherence to the initial SSC guidelines treatment measures. In our study, no patient received the complete 1 -h treatment bundle during the pre-protocol period, compared to a 62% adherence to this bundle in the post-protocol phase. Paul et al. described a similar increase in the adherence to PALS septic shock guidelines from 0% to 50% after 2 months and to 100% after 11 months of the implementation of a protocol associated with the maintenance of quality improvement interventions. Other authors have found similar results.25, 26

We also demonstrated a significant improvement in the compliance with each of the three measures that compose the 1 -h bundle. Our confidence intervals (CIs) were high, probably due to the small number of patients, and our study is retrospective and developed in only one center, but if we consider just the lower limits of the CIs we still have enormous effects: a difference almost 4 times greater in timely initial fluid resuscitation and blood culture collection before antibiotic administration, and almost 9 times greater in antibiotic administration in the first hour, demonstrating the positive impact of the protocol implementation. A recent meta-analysis studying the effect of improvement programs on compliance with sepsis bundles also identified better compliance with complete 1 -h bundle (OR 4.1) and its three measures: fluid resuscitation (OR 3.2), blood culture (OR 2.1), and antibiotics (OR 2.2).

These results reinforce the effectiveness of combining quality improvement interventions with protocol implementations. It is possible that our good results reflect the adoption of several improvement cycles during the implementation of the sepsis protocol, which included consensus meetings, team activation flow, availability of medications, periodic training of the team, and internal dissemination of the results, among other measures. And this may be considered a type of “Hawthorne effect”, which could explain part of our good results. Several studies corroborate the effectiveness of this methodology, indicating that quality interventions positively interfere with compliance with the 1 -h measures for the management of sepsis.17, 19, 25, 26

Our study also demonstrated that the implementation of the SSC protocol was able to significantly reduce the time interval between sepsis recognition and the outcomes fluid resuscitation and antibiotic administration. Approximately two thirds (67.4%) of the patients received timely initial fluid resuscitation within the first hour in the post-protocol phase, representing a 92% reduction in the median time to onset of fluid resuscitation (152 min to 12 min), an effect even greater than that reported by Cruz et al., who described a reduction from 72−22 min. Regarding the start of antibiotics in the first hour, we observed that 75% of patients received the first dose within 45 min during the protocol phase, which represented a reduction in the antibiotic administration median time from 137−30 min. This was achieved by a set of measures, of which we highlight the awareness of the multidisciplinary team, the possibility of manual prescription being accepted by the pharmacy, and the availability of antibiotics in decentralized posts. Studies have reported similar results, with a reduction ranging from 20 to 30% in antibiotic administration median time,25, 27 and a study in adults has shown that the early antibiotic use is more important than give volume to reduce sepsis mortality.

Finally, our study showed a significant drop in mortality from sepsis, which seems to be associated with the implementation of the protocol added to the reinforcement measures and the quality improvement cycles that were an inseparable part of the protocol. There was a four-fold decrease in the risk of death from sepsis and an absolute risk reduction of 9%. Similar decreases were demonstrated in the meta-analysis by Damiani et al. (OR 0.7) and in a study conducted in Canada, which indicated an absolute risk reduction in sepsis mortality of 13%. In the pediatric population, a prospective multicenter study conducted in Thailand evaluated sepsis treatment following the guidelines of the Surviving Sepsis Campaign compared to a control group. The authors reported a significantly lower mortality rate in the intervention group (14% x 37%, OR 0.58). The significant increase in compliance with the 1 -h treatment bundle in our study may be related to the observed reduction in sepsis mortality. Evans et al. also demonstrated that adherence to the 1 -h bundle was associated with a reduction in hospital mortality in pediatric patients treated in an emergency department.[15]

This study has several limitations. First, the retrospective design could increase the risk of participation and information bias. We could not rely on sepsis diagnostic codes to identify all patients with sepsis. Thus, most cases, especially in the pre-protocol period, were identified through a retrospective search for patients undergoing antibiotic regimens recommended for sepsis and the presence of the 2005 international sepsis criteria. We recognize that these criteria may not be the most appropriate, but other authors have also used them, since the Sepsis 3 criteria for adults are based on an organ dysfunction score not validated in pediatrics, and also the 2020 SSC guidelines for children recognize that there is still no adequate definition, although they recommended the inclusion of organ dysfunction criteria for the characterization of sepsis in pediatrics. It is possible that some patients with sepsis criteria were not included if sepsis had never been suspected and antibiotic therapy had never been initiated, but this would be very unlikely, considering the severe evolution of sepsis without antibiotics. In addition, there is a possibility of inaccuracy of data recorded in medical charts, such as the moment when patients met the sepsis criteria, which could overestimate or underestimate the number of sepsis cases and the time-related outcomes. On the other hand, the care team remained stable throughout the study period, with low turnover rates, which may have contributed to minimize data recording errors. Another limitation was that the criteria used for identifying patients with sepsis are considered highly sensitive. As there are no other physiological or laboratory data to confirm the diagnosis of sepsis, especially in the pre-protocol period, the possibility that patients without sepsis were included and died from some other cause could have increased the percentage of deaths in the pre-protocol phase, but this cannot be categorically stated with this type of study. However, the same diagnostic criteria were used in both pre- and post-protocol implementation periods and a real reduction in mortality was demonstrated, although it is important to emphasize that the application of the diagnostic criteria for sepsis was much more precise in the post-phase, after all the training and preparation of the team. Other important issues are the before and after design and the long study period (almost 5 years). We cannot rule out the influence of external factors, such as information on sepsis diagnosis and management, acquired in medical conferences, and training in other workplaces, which may have contributed to the good results observed in our study, regardless of the protocol implementation. It is also important to note that our study excluded patients with febrile neutropenia, a population with a high risk of sepsis and higher mortality, and we also did not have the severity scores (PIM or PRISM) of the patients in the two phases of the study. On the other hand, the high odds ratios observed suggest a high strength of association with the measures adopted. Finally, our hospital is a referral center for patients with serious illnesses and comorbidities, who are a minority in general pediatric hospitals. The international pediatric consensus12, 13 applies to the general pediatric population, so future studies are necessary to confirm the impact of the implementation of a sepsis protocol in a general pediatric population. Also, a consensus must be developed that includes sepsis diagnosis and treatment in subgroups of pediatric patients, especially with diseases that may cause immunosuppression. Despite these limitations, the results reported are significant and should serve as a stimulus to the implementation of protocols for the management of complex and potentially fatal diseases such as sepsis.

Conclusions

Even if we considered the low precision of some estimates, the lower limits of the Confidence Intervals of our data demonstrated that the implementation of the Pediatric SSC protocol, accompanied by quality improvement intervention measures, can improve sepsis recognition, compliance with the 1 -h treatment bundle, and the time interval to fluid resuscitation and antibiotics, as well as reduce sepsis mortality. We believe that the use of quality tools to identify problems, promote solutions and monitor results was essential to obtain the good results achieved with the protocol implementation.

Conflict of interest

The authors declare no conflicts of interest.