Checklist for Early Recognition and Treatment of Acute Illness and Injury: An Exploratory Multicenter International Quality-Improvement Study in the ICUs With Variable Resources.

OBJECTIVES: To determine whether the "Checklist for Early Recognition and Treatment of Acute Illness and Injury" decision support tool during ICU admission and rounding is associated with improvements in nonadherence to evidence-based daily care processes and outcomes in variably resourced ICUs. DESIGN, SETTINGS, PATIENTS: This before-after study was performed in 34 ICUs (15 countries) from 2013 to 2017. Data were collected for 3 months before and 6 months after Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation.
INTERVENTIONS: Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation using remote simulation training.
MEASUREMENTS AND MAIN RESULTS: The coprimary outcomes, modified from the original protocol before data analysis, were nonadherence to 10 basic care processes and ICU and hospital length of stay. There were 1,447 patients in the preimplementation phase and 2,809 patients in the postimplementation phase. After adjusting for center effect, Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation was associated with reduced nonadherence to care processes (adjusted incidence rate ratio [95% CI]): deep vein thrombosis prophylaxis (0.74 [0.68-0.81), peptic ulcer prophylaxis (0.46 [0.38-0.57]), spontaneous breathing trial (0.81 [0.76-0.86]), family conferences (0.86 [0.81-0.92]), and daily assessment for the need of central venous catheters (0.85 [0.81-0.90]), urinary catheters (0.84 [0.80-0.88]), antimicrobials (0.66 [0.62-0.71]), and sedation (0.62 [0.57-0.67]). Analyses adjusted for baseline characteristics showed associations of Checklist for Early Recognition and Treatment of Acute Illness and Injury implementation with decreased ICU length of stay (adjusted ratio of geometric means [95% CI]) 0.86 [0.80-0.92]), hospital length of stay (0.92 [0.85-0.97]), and hospital mortality (adjusted odds ratio [95% CI], 0.81 (0.69-0.95).
CONCLUSIONS: A quality-improvement intervention with remote simulation training to implement a decision support tool was associated with decreased nonadherence to daily care processes, shorter length of stay, and decreased mortality.

The delivery of high-quality evidence-based medicine is expected to minimize complications and improve outcomes in critically ill patients. However, quality-improvement studies in ICU have yielded conflicting results (1–3). Most were performed in high-income country (HIC) ICUs, and no large-scale global quality-improvement efforts have been undertaken in low- and middle-income countries (LMICs) (4) despite their appeal for resource-limited settings where the burden of critical illness is high, and multiple barriers preclude evidence-based care (5). In LMICs, however, a systematic approach to error prevention with the use of checklist has been shown to improve patients’ safety in surgical theaters (6).

To facilitate timely and accurate best practice delivery in critically ill patients in ICUs with variable resources, an international collaboration developed the Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN), a clinical decision support tool that provides evidence-based checklists for structured ICU admission and rounding. Based on user-centered design, we developed CERTAIN that is fast, easy to use and applicable in diverse clinical settings (7, 8). We hypothesized that care assisted by CERTAIN would improve the processes of care and outcomes of critically ill patients in ICUs with variable resources.

MATERIALS AND METHODS

Design and Setting

This was a pragmatic, prospective, before-after quality-improvement study conducted between November 1, 2013, and December 31, 2017. In the preimplementation phase (phase 1, 3 mo or minimum 50 patients per ICU, whatever was reached first), we collected baseline data including ICU characteristics, demographics, baseline severity, comorbidities, limitations on life support interventions, daily care processes, and clinical outcomes of patients admitted to the participating ICUs. Following structured CERTAIN implementation (phase 2), the same variables were collected in the postimplementation phase (phase 3, 6 mo or minimum 150 patients per ICU, whatever was reached first). A detailed description of the methods is published (8). The study was approved by the research ethics boards of all participating hospitals.

The study included ICUs in 10 LMICs and five HICs, across five continents (eTable 1, Supplemental Digital Content 1, http://links.lww.com/CCM/G189); country income status was defined by the World Bank (9). Those ICUs was approached to participate based on previous collaboration with the investigators and through contacts established through members of the European Society of Intensive Care Medicine global working group and American Thoracic Society international health committee. The participating ICUs had variable size (median 12 beds, range, 5–52) and resources; a detailed description is given in eTable 1 (Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Intervention Details and Phases of Implementation

CERTAIN is a web-based decision support tool displaying relevant clinical information with a systematic approach. It incorporates evidence-based checklists, algorithms, and educational modules on performing critical procedures and has been reported previously (8). CERTAIN consists of two modules: an admission module and a rounding module (eFigs. 1–3, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). A mobile version and paper version were also provided in case of problems with internet connection. Items of admission and rounding checklists are presented in Figures and . The PDF format of the admission checklist, rounding checklist, and individual syndrome cards are also available for download on the CERTAIN website: https://www.icertain.org/library. The tool was tested in a simulated environment before the implementation phase (10, 11).

Checklist for Early Recognition and Treatment of Acute Illness and Injury admission module checklist abbreviated (full version is in the supplement, Supplemental Digital Content 2, http://links.lww.com/CCM/G190). BP = blood pressure, BUN = blood urea nitrogen, CHF = congestive heart failure, Cr = creatinine, Billi = bilirubin, DNI = do not intubate, DNR = do not resuscitate, ECG = electrocardiogram, Glu = glucose, Hb = hemoglobin, HR = heart rate, INR = international normalized ratio, Lam = lactate, Plt = platelets, RR = respiratory rate, SARS = severe acute respiratory syndrome, SC = subcutaneous, SpO2 = blood oxygen saturation, TB = tuberculosis, Temp = temperature, UO = urine output, US = ultrasound, V tach = ventricular tachycardia.

Checklist for Early Recognition and Treatment of Acute Illness and Injury rounding module checklist abbreviated (full version is in the supplement, Supplemental Digital Content 3, http://links.lww.com/CCM/G191). HEM = hematology, Spont. = spontaneous, vent. = ventilation.

CERTAIN implementation was done through an education program that engaged with local teams including at least three ICU members. After 2–4 weeks of access to online curriculum (slide presentations, published papers), the team conducted a follow-up online remote training via simulation coaching and debriefing (eFig. 4, Supplemental Digital Content 1, http://links.lww.com/CCM/G189) (8, 11). Following a train-the-trainer session, the local trainer was certified to train their local staff. Once local physicians and nurses completed the training, the participating center proceeded to clinical implementation. The core of the intervention was a structured approach to admission and daily rounding by using the checklist to prompt clinicians to follow best care practices. At the discretion of individual ICUs, a dedicated ICU team member used the CERTAIN checklist to prompt other team members to review checklist items within the tool during admission and rounding.

To enter the postimplementation phase, all centers were required to meet greater than 80% adherence to both the rounding and admission checklists for 4 consecutive weeks during the implementation phase. Adherence was tracked weekly through self-reported online form. CERTAIN was implemented sequentially in each ICU, so that by the end of the study, all ICUs had received the intervention (eFigs. 5 and 6, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Patients and Data Collection

In both pre- and postimplementation phases, data were collected prospectively on consecutive patients (≥ 18 yr) admitted to the study ICUs. Readmissions, patients who were admitted for simple monitoring, patients who had a planned ICU admission for routine postoperative surveillance lasting less than 24 hours after uncomplicated surgery (this and previous criterion as defined by the site investigators), and patients who were transferred from an ICU outside the participating hospital were excluded.

Data were collected using the secure Research Electronic Data Capture system (12). All data collectors were trained by the coordinating team and via online tutorials. The coordinating center audited data collection and met investigators regularly via monthly videoconference.

Study Outcomes and Definitions

The study’s coprimary outcomes include the following: 1) nonadherence to daily processes of care and 2) ICU and hospital length of stay (LOS). Secondary outcomes include 1) ICU mortality, 2) hospital mortality, 3) 28-day mortality, 4) blood product utilization, 5) infections, and 6) survival time. We observed the nonadherence to processes on days 0 (the day of ICU admission), 1, 2, 3, 7, 14, and 21, as long as the patient remained in the ICU. Nonadherence was recorded for the following 10 daily care processes: 1) deep vein thrombosis (DVT) prophylaxis, 2) peptic ulcer prophylaxis, 3) oral care, 4) head of bed (HOB) elevation to at least 30° above horizontal, 5) spontaneous breathing trial (SBT), 6) family conference discussion (for patients on mechanical ventilation), 7) assessment of central venous catheter (CVC) removal (for patients with CVC), 8) assessment for urinary catheter removal (for patients with urinary catheter), 9) assessment to continue or discontinue current antimicrobials (for patients receiving antimicrobial therapy), and 10) assessment to continue or discontinue current sedation (for patients receiving sedation). Nonadherence to these daily processes of care was quantified as an incidence rate, defined as the ratio of the number of observations of not receiving basic care procedures among eligible patients (events) to the number of total observations of in which the specific intervention was indicated (exposure) expressed per 1,000 days of specific intervention (eTable 3, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

The published study protocol defined the primary outcome as adherence to care practices; we changed this to nonadherence because it was a “rare” event, and we used Poisson regression for analysis. Modifications from the original protocol were done before data analysis and are presented in eTable 7 (Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Blood product utilization was measured by documenting RBC, fresh frozen plasma (FFP), and platelet transfusions. For infections, we recorded incidence rates of ventilator-associated pneumonia (VAP), catheter-related bloodstream infections (CRBSIs), and urinary catheter infections. Although we provided standard operating definitions for the infection outcomes (13), the final diagnosis was based on medical records; we did not request microbiologic data to validate this diagnosis. For ICU and hospital LOS, the interval between the ICU admission date and the ICU or hospital discharge date was calculated in days. Patient status at ICU, hospital discharge, and at 28 days was assessed by reviewing the medical records and by postdischarge telephone contact (for discharged patients), respectively.

We did not assess adherence to some of the prespecified outcomes (e.g., shock resuscitation and sepsis treatment) due to limited resources for real-time data collection (eTable 5, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). The incidence rates of VAP, CRBSI, and urinary catheter infections, and RBC, FFP, and platelet transfusions were added after the study protocol publication but before study completion.

Statistical Analysis

Detailed statistical methods are presented in the Supplementary Digital Content (Supplemental Digital Content 1, http://links.lww.com/CCM/G189). Categorical data are reported as counts and proportions and continuous data as mean (sd) or median (interquartile range [IQR]).

The study encompasses six sets of clinical outcomes that were measured before and after CERTAIN implementation. These outcomes were studied in all centers. Assessments of CERTAIN’s effect were performed by developing linear, logistic, and Poisson models as the data required. These models were used to estimate ratios that expressed the effect of CERTAIN. The model’s variables included CERTAIN implementation, center (to account for the clustering of patients within center), and patient characteristics (to account for imbalances between the pre/post implementation phases). For example, in the model for mortality, the regression coefficient (sd) for the CERTAIN implementation variable was used to estimate the adjusted odds ratio (aOR) and its CI. The p values are based on the likelihood ratio test using nested and full models. All tests are two sided, and a p value of less than 0.05 was considered statistically significant. All statistical analysis was conducted using the R statistical software, Version 3.4.(R Foundation for Statistical Computing, Vienna, Austria. URL http://www.R-project.org/.)

Revised Standards for Quality Improvement Reporting Excellence (SQUIRE 2.0) were used as a framework for conducting the study (14, 15).

RESULTS

Study Flow and Patient Characteristics

A total 55 centers were approached, of which 46 accepted to participate and 34 centers completed the study. A total of 4,256 patients completed the study: 1,447 in preimplementation and 2,809 in postimplementation phases (Fig. ). Twelve centers did not complete the study: eight centers dropped out during preimplementation phase, three during implementation phase, and one center in final postimplementation phase. Reasons for drop-out included lack of resources and change in leadership leading to a shift in priorities. Baseline demographics and severity of illness of patients in pre- and postimplementation phases were similar. More patients in the postimplementation phase were admitted from home, whereas patients in preimplementation phase had more comorbidities and more limitations of life support interventions at ICU admission (Table ). The median duration of ICU study participation was 96 weeks (IQR, 54–139 wk) (eTable 4, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Study flow. IRB = institutional review board.

Primary Outcomes

Following implementation of CERTAIN, there was a significant decrease in nonadherence to eight of 10 targeted daily care processes compared with preimplementation period (Table and Fig. ). There were 2,840 mechanically ventilated patients resulting in 10,069 observation days (3453 in preimplementation and 6,616 in postimplementation phases).

Checklist for Early Recognition and Treatment of Acute Illness and Injury (CERTAIN): change in nonadherence to daily care processes before and after CERTAIN implementation. “Mech vent” denotes that those care practices were only measured in mechanically ventilated patients. CVC = central venous catheter, DVT = deep vein thrombosis, HOB = head of bed, SBT = spontaneous breathing trial, UC = urinary catheter.

Compared with the preimplementation period, the incidence of nonadherence to DVT and peptic ulcer prophylaxis was significantly reduced in the postimplementation period (adjusted incidence rate ratio [aIRR], 0.74; 95% CI, 0.68–0.81 and 0.46; 95% CI, 0.38–0.57, respectively). There was a decrease in nonadherence to daily assessment of CVCs (from 647 to 566 per 1,000 CVC days; aIRR, 0.85; 95% CI, 0.81–0.90) and urinary catheters (674 vs 533 per 1,000 urinary catheter days; aIRR, 0.84; 95% CI, 0.80–0.88) (Table 2). There was also an improvement in adherence to daily assessment of medications. The incidence rate for not assessing the need for antimicrobials was reduced from 340 to 219 per 1,000 observation days (aIRR, 0.66; 95% CI, 0.62–0.71). Omissions in daily assessment of sedation need decreased from 359 to 212 per 1,000 observation days (aIRR, 0.62; 95% CI, 0.57–0.67) (Table 2). The intervention was also associated with reduced nonadherence to the best practices of conducting daily SBTs and holding family conferences. There was no change in nonadherence to daily oral care and HOB elevation.

TABLE 2.

The Incidence Rates of Nonadherence to Daily Care Processes Before and After Checklist for Early Recognition and Treatment of Acute Illness and Injury Implementation

Total 4,256	Pre Intervention, N = 1,447				Post Intervention, N = 2,809
Observed Event	Incidence Rate (95% CI)	Observation Days, n	Observed Events, n	Patients, n	Incidence Rate (95% CI)	Observation Days, n	Observed Events, n	Patients, n	Adjusted for Center Effects Incidence Rate Ratio	p
Mechanical ventilation
	Per 1,000 ventilator days				Per 1,000 ventilator days
No deep vein thrombosis prophylaxis	268 (251–286)	3,453	926	966	200 (189–211)	6,616	1,322	1,874	0.74 (0.68–0.81)	< 0.001
No peptic ulcer prophylaxis	55 (48–64)	3,453	193	966	28 (24–32)	6,616	185	1,874	0.46 (0.38–0.57)	< 0.001
No documented assessment of spontaneous breathing trial	563 (539–589)	3,453	1,946	966	450 (434–467)	6,616	2,980	1,874	0.81 (0.76–0.86)	< 0.001
No documented family conference/ discussion	416 (395–438)	3,453	1,437	966	361 (347–376)	6,616	2,391	1,874	0.86 (0.81–0.92)	< 0.001
No daily oral care	45 (38–52)	3,453	155	966	42 (38–48)	6,616	281	1,874	0.94 (0.77–1.16)	0.60
No head of bed elevation at 30°	39 (33–46)	3,453	135	966	37 (32–42)	6,616	244	1,874	1.00 (0.81–1.23)	0.98
CVCs
	Per 1,000 CVC days				Per 1,000 CVC days			1,896
No documented assessment for CVC removal	647 (622–673)	3,922	2,538	990	566 (549–583)	7,308	4,136	1,896	0.85 (0.81–0.90)	< 0.001
Urinary catheters,
No documented assessment for urinary catheter removal (per 1,000 urinary catheter days)	674 (653–697)	5,268	3,553	1,322	564 (549–578)	10,228	5,766	2,609	0.84 (0.80–0.88)	< 0.001
Antimicrobials and sedation
	Per 1,000 d of use				Per 1,000 d of use
No documented assessment to continue or discontinue current antimicrobials	340 (324–356)	5,168	1,758	1,310	219 (209–228)	9,497	2,079	2,498	0.66 (0.62–0.71)	< 0.001
No documented assessment to continue or discontinue current sedation	359 (338–381)	3,049	1,092	941	212 (201–224)	5,436	1,251	1,895	0.62 (0.57–0.67)	< 0.001

CVC = central venous catheter.

The mean ICU LOS was 6.5 days (sd 3.0 d) in pre- and 5.8 days (sd 2.8 d) days in postimplementation phases, and the mean hospital LOS was 14.9 days (sd 3.1 d) in preimplementation versus 14.6 days (sd 2.8 d) in postimplementation phases (adjusted ratio of geometric means [aRoGM], 0.86; 95% CI, 0.80–0.92, and 0.92; 95% CI, 0.85–0.97, respectively). Results were similar after adjusting for baseline patient covariates, income level, and center (aRoGM, 0.86; 95% CI, 0.80–0.92 and 0.92; 95% CI, 0.86–0.98, respectively) (Table ).

Secondary Outcomes

The overall (combined pre- and postimplementation phases) ICU, hospital, and 28-day mortality were 24.8%, 29.8%, and 31.7%. There was a significant ICU mortality reduction from 28.8% (preimplementation phase) to 23.5% (postimplementation phase) (aOR, 0.84; 95% CI, 0.71–0.99). Similarly, the hospital mortality reduction was 34.2% to 28.8% (aOR, 0.81; 95% CI, 0.69–0.95). Finally, the 28-day mortality reduction was 36.9% to 30.9% (aOR, 0.82; 95% CI, 0.70–0.95). The reductions in ICU, hospital, and 28-day mortality remained significant after adjusting for baseline imbalances, income status, and center characteristics (aOR, 0.74; 95% CI, 0.63–0.87; 0.74, 0.64–0.86; 0.70, 0.66–0.89, respectively) (Table 3).

TABLE 3.

The patient outcomes before and after Checklist for Early Recognition and Treatment of Acute Illness and Injury Implementation

Total 4,256	Pre Intervention, N = 1,447			Post Intervention, N = 2,809			Effect Measure (95% CI)	p
Length of stay
	Geometric Mean (sd), d			Geometric Mean (sd), d			Adjusted Ratio of Geometric Means
ICU	6.5 (3.0)			5.8 (2.8)			0.86 (0.80–0.92)	< 0.001
Hospital	14.9 (3.1)			14.6 (2.8)			0.91 (0.85–0.97)	0.003
Mortality
	Mortality (%)	Death	Patient, n	Mortality (%)	Death	Patient, n	Adjusted OR	p
ICU mortality	28.8 (26.5–31.2)	406	1,409	23.5 (21.9–25.1)	648	2,762	0.84 (0.71–0.99)	0.041
Hospital mortality	34.2 (31.7–36.7)	477	1,396	28.8 (27.1–30.5)	793	2,757	0.81 (0.69–0.95)	0.01
28 d mortality	36.9 (34.4–39.5)	504	1,367	30.9 (29.2–32.7)	845	2,731	0.82 (0.70–0.95)	0.012

OR = odds ratio.

From 2,886 patients with CVCs (68% of total; 990 (68%) in preimplementation and 1,896 (67%) in postimplementation phases), 97 patients (3.4%) had CRBSI. There was a reduction of CRBSI in postimplementation phase (4.4% vs 2.8%; aOR, 0.58; 95% CI, 0.38–0.90). Urinary catheter infection was present in 58 of 1,322 patients (4.39%) in preimplementation and 134 of 2,609 (4.02%) in postimplementation phases (aOR, 0.87; 95% CI, 0.62–1.23). The risk of VAP remained unchanged before and after CERTAIN implementation (18.7% vs 20.0%; aOR, 1.02; 95% CI, 0.80–1.28). The incidence rate of RBC and platelet transfusion did not differ between the study phases. The incidence rate of FFP transfusion was lower in postimplementation compared with preimplementation phase (aIRR, 0.75; 95% CI, 0.66–0.85) (eTable 8, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

Post Hoc Analyses

In a post hoc subgroup analysis according to World Bank–defined country income status, the improvements were more pronounced in LMICs (reduction in nonadherence in 8/10 processes) compared with HICs (reduction in nonadherence in 6/10 processes) (eTables 5 and 6, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). Omissions in adherence to SBT, assessment for CVC and urinary catheter removal, and antimicrobial and sedation use decreased in both HIC and LMICs, whereas omissions in DVT and peptic ulcer prophylaxis and holding family conferences were reduced only in LMICs. Omissions in adherence to HOB elevation decreased only in HICs. The implementation of CERTAIN was associated with reduction in incidence rates of CRBSI and FFP transfusions in LMICs but not HICs (eTables 5 and 6, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

In a related post hoc analysis, significant interaction between income status and CERTAIN implementation effects on ICU, hospital, and day 28 mortality was observed (p < 0.001, p = 0.004, p = 0.002 respectively). After adjusting for baseline imbalances, the ICU mortality (LMIC: aOR, 0.63; 95% CI, 0.53–0.75; HIC: aOR, 1.18; 95% CI, 0.83–1.68), hospital mortality (LMIC: aOR, 0.65; 95% CI, 0.55–0.77, HIC: aOR, 1.03; 95% CI, 0.75–1.43), and 28-day mortality (LMIC: aOR, 0.67; 95% CI, 0.56–0.79, HIC: aOR, 1.19; 95% CI, 0.87–1.63) were reduced only in LMICs. The survival analysis stratified according to income status reported similar differences in CERTAIN effect (eFig. 8, Supplemental Digital Content 1, http://links.lww.com/CCM/G189).

DISCUSSION

In this multinational study of ICUs with variable resources, prompting the ICU clinical team with CERTAIN admission and rounding checklists was associated with lower nonadherence to most daily care processes, reduced LOS, and reduced mortality. To our knowledge, this is the first global effort to introduce a checklist into everyday clinical practice to improve adherence to ICU daily care processes and patient outcomes. We demonstrated that CERTAIN implementation is associated with significant improvements of evidence-based care delivery.

CERTAIN implementation was associated with improvement of eight of 10 practices with no improvement in HOB elevation and daily oral care practice. The baseline adherence to many processes of care was high, with the greatest improvement in processes with lower adherence in preimplementation phase. Previous studies on checklist use in ICUs have yielded conflicting results (1, 2, 6, 16–20). Improvements were seen in HOB elevation, daily interruption of sedative infusions, peptic ulcer disease and deep venous thrombosis prophylaxis, oral care for ventilated patients, electrolyte repletion, initiation of physical therapy, and documentation of restraint orders (21, 22). Overall, the results were not consistent across studies (23–25). This variation in effect may be due to differences in checklist selection and outcome definition. The way in which checklists are implemented may also influence the results, as active prompting has been shown to be superior to a checklist alone (20). Most studies, including this one, used a before-after design and thus are prone to secular trends. Importantly, previous studies were mostly performed in HICs that have more resources for quality-improvement interventions and an established organizational safety culture. Notably, daily care processes in our study improved more in LMICs than HICs, which may relate to higher adherence to baseline processes of care in HICs but also fewer centers enrolled from HICs.

In LMIC ICUs, lack of structured training and lack of physicians are major barriers to implementing evidence-based medicine (26). A systematic approach and provision of low-cost basic critical care procedures can minimize death and reduce costly complications in all environments (27, 28). Simple interventions such as checklists improved outcomes in surgical and trauma patients globally without significantly increasing costs (6, 29). So far, data on efficacy of such quality-improvement processes in ICUs of developing countries are scarce. Recently, a quality-improvement initiative in sepsis was associated with improved compliance to quality indicators and reduced mortality in a middle-income country (30). Advances in information technology have yielded opportunities for online remote training and quality improvement without costly on-site education. Such use of collaborative network infrastructure has already shown to improve adaptation of care practices in HIC community ICUs (3).

We observed a significant decrease in number of CVC infections in the postimplementation phase with no effect on prevention of VAP and urinary catheter infections. Studies on preventing ICU infections with quality-improvement interventions have shown different results (16, 19, 31). This heterogeneity of effect may be due to nonuniform definitions of ICU infections. Mortality was significantly lower in the post-CERTAIN phase. In a stratified analysis according to income status, the mortality reduction was present only in LMICs and remained significant after adjusting for baseline imbalances between pre- and postimplementation groups. In contrast, a randomized trial to introduce prompting and checklists did not decrease hospital mortality and ICU infections in Brazilian ICUs (2) but did show modest improvements in four of seven processes of care. The main difference is that in our study, the intervention (training and checklist) not only included a rounding checklist (CERTAIN rounds) but also a structured admission and resuscitation checklist (CERTAIN admission). Similar to our findings, a nonrandomized evaluation of the global implementation of a surgical checklist was associated with marked reductions in mortality and postoperative complications in economically diverse group of hospitals (6), but the same effect was not observed in a HIC setting (32).

Our study has several limitations. Before-after studies are prone to confounding by secular improvements and Hawthorne effect. Several aspects of management may have changed over time; for example, transfusion triggers may have declined. Therefore, cause and effect cannot be reliably determined. Second, in this pragmatic study, data were collected by bedside clinicians rather than dedicated study coordinators. We encouraged adherence to the intervention throughout the study period and measured it via self-reporting rather than direct observation. Third, the baseline characteristics of pre- and postintervention groups differed. The preimplementation cohort had more comorbidities and more limitations on life-sustaining measures, both of which could contribute to the mortality differences seen. Nevertheless, the association of CERTAIN with decreased LOS and mortality remained after adjustment for the baseline imbalances. Although the statistical analysis may be able to control for some of this confounding, residual confounding may persist. Fourth, the diagnosis of CVC and urinary catheter infections and VAP were based on medical records rather than adjudication including microbiological data. Fifth, interventions such as catheter insertion bundles, nutrition, or decubitus ulcer prevention were not examined. The evaluated daily care processes were mostly addressed by the CERTAIN rounding checklist, and the impact of the admission checklist and decision support algorithms on outcomes could not be reliably assessed in the absence of validated measures. Sixth, ICU-specific factors including staffing, number of beds, baseline performance, and prior internal quality-improvement interventions may have influenced the results. The time frame in which the ICUs adopted and implemented the intervention differed significantly due to the duration of the study period. Challenges in CERTAIN implementation are discussed elsewhere (33). Only those hospitals motivated to improve quality care were committed time and resources to complete the study (eFig. 7, Supplemental Digital Content 1, http://links.lww.com/CCM/G189). However, the observed association persisted after adjustment for center. The patients who were transferred from an ICU outside the participating hospital were excluded from this study; the results may not be applicable on those patients. Last, some of the elements of SQUIRE guidelines including cost and strategic trade-offs, opportunity cost, and sustainability were not analyzed. Given these limitations, additional carefully designed studies are required to validate CERTAIN association with improved outcomes.

Despite these limitations, the results of this study show that systematic checklist implementation is feasible even in resource-limited settings and is associated with improvements in processes of care. To further facilitate CERTAIN use in daily work flow, we enabled translation into local language and developed a paper version of the tool. In addition, we have conducted educational courses (domestics, international, and online programs) and developed a train-the-trainer course to share our experience globally (www.icertain.org).

CONCLUSIONS

The implementation of a structured electronic decision tool with checklists in international ICUs has shown an association with reductions in nonadherence to basic care processes, CRBSI risk, and mortality.

ACKNOWLEDGMENT

In memoriam of Dr. Bekele Afessa, one of the pioneers of critical care practice improvement whose teachings inspired this work. We would like to thank Dr. Chris Farmer, Dr. Terri Clemmer, Dr. Atul Gawande, and Dr. Ken Hillman for helpful advice during study design and conduct. We thank Michael Malinchoc, MS, for performing statistical analysis. We thank the participating ICUs and their teams for their enthusiastic work and contribution to this project.

TABLE 1.

Baseline Characteristics of Patients in Pre- and Postimplementation Groups

	Pre Implementation	Post Implementation	p
	N = 1,447	N = 2,809
Age, median (IQR)	62 (46–75)	62 (47–74)	0.92
Gender, female, n (%)	588 (40.6)	1169 (41.6)	0.56
Hospital admission source, n (%)			< 0.001
Home	680 (47.0)	1605 (57.2)
Nursing home	35 (2.4)	39 (1.4)
ED	444 (30.7)	791 (28.2)
Outside hospital ED	228 (15.8)	280 (10.0)
Other	59 (4.1)	91 (3.2)
Life support limitation, n (%)	150 (10.4)	137 (4.9)	< 0.001
Sequential Organ Failure Assessment score, median (IQR)	6 (4–10)	6 (4–9)	0.07
Mechanical ventilation, n (%)	822 (57.0)	1680 (60.4)	0.04
Use of vasoactive medications, n (%)	583 (40.7)	1111 (40.1)	0.75
Antimicrobial medication, n (%)	1198 (83.6)	2256 (81.8)	0.17
Comorbidities, n (%)
Congestive heart failure	270 (18.7)	376 (13.4)	< 0.001
Cardiac arrhythmias	201(13.9)	296 (10.5)	0.001
Valvular disease	93 (6.4)	110 (3.9)	< 0.001
Pulmonary circulation disorders	135 (0.9)	193 (0.7)	0.05
Hypertension	643 (44.4)	1204 (42.9)	0.34
Neurologic disorders	215 (14.9)	274 (9.7)	< 0.001
Diabetes	384 (26.5)	713 (25.3)	0.66
Hypothyroidism	48 (3.3)	111 (3.9)	0.34
Renal failure	229 (15.8)	359 (12.8)	0.007
Liver disease	111 (7.7)	142 (5.1)	< 0.001
Peptic ulcer disease excluding bleeding	40 (2.7)	48 (1.7)	0.03
AIDS	9 (0.6)	12 (0.4)	0.53
Lymphoma	15 (1.0)	49 (1.7)	0.10
Metastatic cancer	64 (4.4)	140 (5.0)	0.46
Solid tumor without metastasis	80 (5.5)	196 (7.0)	0.08
Rheumatoid arthritis/collagen vascular disease	29 (2.0)	55 (2.0)	1.00
Coagulopathy	54 (3.7)	81 (2.9)	0.16
Obesity	124 (8.6)	123 (4.4)	< 0.001
Weight loss	62 (4.3)	79 (2.8)	0.01
Fluid and electrolyte disorders	169 (11.7)	238 (8.5)	< 0.001
Blood loss anemia	50(3.5)	102 (3.1)	0.84
Deficiency anemia	63 (4.3)	61 (2.2)	< 0.001
Alcohol abuse	65 (4.5)	90 (3.2)	0.04
Drug abuse	23 (1.6)	46 (1.6)	1.00
Psychosis	24 (1.7)	40 (1.4)	0.64
Depression	59 (4.1)	75 (2.7)	0.02
Other	390 (26.9)	703 (25.0)	0.19
None	88 (6.1)	362 (12.9)	< 0.001

ED = emergency department, IQR = interquartile range.

p based on χ2 when comparing categorical variables or Kruskal-Wallis rank-sum test when comparing quantitative variables.