Causal Analysis of World Health Organization's Surgical Safety Checklist Implementation Quality and Impact on Care Processes and Patient Outcomes: Secondary Analysis From a Large Stepped Wedge Cluster Randomized Controlled Trial in Norway.

OBJECTIVE: We hypothesize that high-quality implementation of the World Health Organization's Surgical Safety Checklist (SSC) will lead to improved care processes and subsequently reduction of peri- and postoperative complications.
BACKGROUND: Implementation of the SSC was associated with robust reduction in morbidity and length of in-hospital stay in a stepped wedge cluster randomized controlled trial conducted in 2 Norwegian hospitals. Further investigation of precisely how the SSC improves care processes and subsequently patient outcomes is needed to understand the causal mechanisms of improvement.
METHODS: Care process metrics are reported from one of our earlier trial hospitals. Primary outcomes were in-hospital complications and care process metrics, e.g., patient warming and antibiotics. Secondary outcome was quality of SSC implementation. Analyses include Pearson's exact χ test and binary logistic regression.
RESULTS: A total of 3702 procedures (1398 control vs. 2304 intervention procedures) were analyzed. High-quality SSC implementation (all 3 checklist parts) improved processes and outcomes of care. Use of forced air warming blankets increased from 35.3% to 42.4% (P < 0.001). Antibiotic administration postincision decreased from 12.5% to 9.8%, antibiotic administration preincision increased from 54.5% to 63.1%, and nonadministration of antibiotics decreased from 33.0% to 27.1%. Surgical infections decreased from 7.4% (104/1398) to 3.6% (P < 0.001). Adjusted SSC effect on surgical infections resulted in an odds ratio (OR) of 0.52 (95% confidence interval (CI): 0.38-0.72) for intervention procedures, 0.54 (95% CI: 0.37-0.79) for antibiotics provided before incision, and 0.24 (95% CI: 0.11-0.52) when using forced air warming blankets. Blood transfusion costs were reduced by 40% with the use of the SSC.
CONCLUSIONS: When implemented well, the SSC improved operating room care processes; subsequently, high-quality SSC implementation and improved care processes led to better patient outcomes.

RCT Entities:   Population Interventions Outcomes

The World Health Organization's (WHO) Surgical Safety Checklist (SSC) has been reported to reduce both morbidity and mortality.[1,2] The SSC was developed to improve teamwork, communication and consistency of care in operating rooms.[3] Enhanced teamwork and communication is one of the mechanisms used to explain SSC effects on patient outcome.[4-6] Facilitators of SSC use that strengthen implementation are reported to be education and training, audit and feedback interventions using local data on actual checklist usage, fostering local champions and leadership, and accountability for compliance.[7] Perceived implementation barriers are design-related issues (including poor local tailoring of items, nonintegration into operating room workflow), lack of structured implementation approach, and resistance from senior clinicians.[7,8]

Precisely how the SSC, or indeed any other checklist that has been evaluated to date, achieves its effectiveness is far from clear. Mechanisms postulated to drive SSC positive effects have been associated with implementation strategies and actual utilization of the checklist.[9,10] Moreover, in studies that find reduced morbidity and mortality,[10-12] quality of SSC implementation is assumed to be an important explanatory mechanism.[9] A large scale study of the SSC effects in Canadian hospitals, including 215,711 procedures, did not find similar results.[13] Nonetheless, the study raised concerns about quality of implementation strategies.[14] In other studies high fidelity to the checklist intervention has proven important for improved patient outcomes.[11,12,15] Taken together, the evidence-base to-date implies that explanatory mechanisms behind effectiveness (or lack thereof, as in the Canadian dataset) are yet to be fully understood.

Lack of understanding of what makes implementation of the SSC effective in some settings, but not in others severely hampers our ability to improve SSC implementation. We remain unaware of which implementation element matters the most and in which settings. In turn, this limits our ability to improve patient outcomes via better application of the SSC. In the WHO SSC implementation guide, hospital leadership, and monitoring of surgical results and complications are recommended to achieve successful implementation.[16] Tracking of process and outcome measures have been encouraged, exemplified by percent of procedures having antibiotics provided at the correct time.[16] Accordingly, the WHO SSC implementation guide rests on Donabedian's approach to clinical quality improvement,[17] in which improved structures enhance care processes; and both structures and care processes, in turn, improve patient outcomes.

This study investigates how exactly the SSC improves patient outcomes via analysis of clinical structures, processes, and outcomes related to SSC implementation in the operating room. The main hypotheses we are testing are:

H1: High-quality implementation of the SSC improves care processes in the operating room;

H2: Improved care processes lead to better patient outcomes;

H3: Improved implementation (fidelity to SSC) leads to improved compliance with critical standards (improved care processes), and improved compliance leads to improved outcomes.

The clinical improvement framework and associated hypotheses we tested, based on Donabedian's approach, are illustrated in Figure 1.

FIGURE 1

A clinical improvement framework and associated study hypotheses, based on Donabedian's approach on structure, process, and outcome.

METHODS

Study Design

Our study was designed as a stepped wedge cluster randomized controlled (RCT) quality service improvement trial in 2009 to 2010.[12] The stepped wedge cluster RCT design is increasingly used to evaluate patient safety interventions that inherently are expected to do more good than harm.[18] The intervention is sequentially introduced to the clusters in a random way at different time points, which is particularly useful when the intervention cannot be delivered to all participants at the same time. Hence, the checklist intervention was provided to 1 cluster at the time.[12] This study was conducted in 2 Norwegian hospitals, a community hospital and a tertiary teaching hospital, and included 5 surgical specialties (orthopedic, cardiothoracic, neurosurgery, urology, and general surgery). The dataset from the original study was further analyzed to search for the effects of process metrics on patient outcomes. Three of the study clusters had such process metrics registered, and were therefore included, hence all other clusters were excluded (SDC 1).

The 3 specialties (clusters of the RCT) were randomly allocated to receive the SSC intervention. Allocation sequences were generated by a draw of numbers into a rank order deciding the roll-out of the checklist intervention. The allocation assessor was blinded for clusters corresponding to the numbers. The SSC implementation started sequentially over 3 to 4 weeks after a 3-month baseline period. The intervention continued for 3 months after all clusters received the intervention. Details of the stepped wedge cluster (RCT) design and the SSC intervention have previously been described.[12,18-20]

The SSC consists of 3 parts, the Sign in before anesthesia induction, the Time out before incision, and the Sign out at the end of the surgical procedure—before transfer to postoperative care unit. The SSC adapted for use in Norwegian operating rooms is presented in SDC 2. In the Norwegian checklist version, items to prevent hypothermia are listed both under the Sign in and under Time out parts.

Use of routinely collected anonymized data was regarded as clinical service improvement by the Regional Committee for Medical and Health Research Ethics (Unique identifier: 2009/561). Hence, approval of the study was given by the hospital privacy Ombudsmen (Ref: 2010/413) and hospital managers.

Outcome Measures

Measures relevant to operating room care processes and patient outcomes were the primary endpoints; quality of SSC implementation was a secondary endpoint.

To avoid possible study biases by introduction of new measurements on process metrics associated with items on the checklist, which could be regarded as competing interventions, we used process metrics that were already being registered as routine practice. Care process metrics were preoperative site marking; actions to sustain normothermia (prewarmed intravenous fluid, prewarmed blankets, forced air warming blankets); and timeliness of infection prophylactic provision of intravenous antibiotics. The latter was categorized into before and after incision, and no antibiotics.

Patient outcomes included surgical infection, surgical wound rupture, cardiac complication, respiratory complication, postoperative bleeding, and intraoperative blood transfusion. We classified the primary endpoints as 0 for no complication and 1 for verified complication. Secondary outcome was blood transfusion costs in USD.

Implementation quality was prospectively measured by the fidelity to actual use of the SSC, defined as compliance with all 3 parts of the checklist. To investigate SSC fidelity impact on patient outcomes as previously shown by Mayer et al,[10] we categorized utilization of the Sign in, Time out and Sign out parts used as: no checklist; one of the checklist parts; combinations of 2 of parts; all 3 parts; and any parts.

Data Collection

Data from all age groups and elective or emergency surgery are included. Surgical procedures which the SSC was not adapted for were excluded (ie, donor surgery). Patient characteristics include age, sex, and comorbidity with the American Society of Anesthesiologists (ASA) classification. Further, data on elective or emergency surgery, type of anesthesia (general vs. regional), surgical procedures as orthopedic, cardiothoracic or neurosurgical, and duration of surgical procedures (knife time) were recorded in the hospital administrative data system as routine practice by clinical staff. Adherence to the SSC was prospectively recorded on a paper form by nurse anesthetists and operating room nurses. All items were marked for each patient, as the SSC parts were carried out. To decide whether it had been used or not, we determined a cut-off requiring more than 60% of items to be registered on the paper version. Additionally, the SSC parts were electronically recorded as used (all items required performed) or not, by the operating room nurse. If there were any discrepancies between paper and electronic recordings of SSC fidelity, the latter was preferred.

To ensure high fidelity to checklist performance, members of our multidisciplinary implementation team were present in the operating rooms. They provided advice through direct guidance and observations on site. Evaluation meetings on checklist fidelity were conducted with the operating teams in the operating theater 2 weeks and 2 months postimplementation of the SSC. Feedback on checklist compliance rates was posted on wall posters outside the operating rooms throughout the study.

Patient complications were assigned International Classification of Diseases – tenth version (ICD-10) codes recorded by surgeons or ward physicians at patients’ discharge from hospital. All outcome data were extracted from hospital administrative databases and quality checked to verify incidence of any recorded complications.[12]

Data Handling

The assessors handling and evaluating data validity were blinded to the randomization of patients and procedures into control and intervention cohorts. To protect the study from information bias, clinicians were not informed as to which study endpoints that were measured. All recovery and postoperative ward staff were not informed about the study, cohorts, or outcome of interest, and performed care as usual. Complications identified through ICD-10 codes and care process data were verified against the patients’ medical records.[12] This study followed the extended CONSORT statement for nonpharmacological randomized trials.[21]

Statistical Analysis

The surgical clusters provided data in all the stepped wedges, being their own controls before and after the introduction of the SSC intervention. Hence, data across the cluster steps before (controls) were compared with the steps after SSC implementation (intervention).[19] Fuller implementation of the SSC (ie, more parts completed) indicates higher fidelity to the intervention.[22] To investigate effect of procedures with highest SSC compliance we also compared controls to intervention procedures with full implementation of the SSC (n = 1743). Patient outcome, patient, and procedure characteristics for the control and intervention stages, and fidelity of checklist implementation (full vs. none) were analyzed using Pearson's exact χ2 test for categorical data, independent samples t test for continuous data, or nonparametric test (Mann–Whitney U test) as appropriate.

Based on our original sample size calculation, a minimum of 1100 patients were required in each one of the control and checklist groups for adequate study power.[12] Intracluster correlation was not calculated as it is considered to have minimal impact on power due to the unidirectional stepped wedge implementation of the intervention.[18] The primary endpoints were modeled with logistic regression. Model I: by SSC fidelity, and in Model II: controlling for patient and procedure characteristics, and process metrics. Analyses were carried out in SPSS version 23.0 (IBM Corp, Armonk, NY), and a 2-sided P value less than 0.05 was considered statistically significant.

RESULTS

Patient Characteristics

Overall, 3702 surgical procedures were included in this stepped wedge cluster RCT, with 1398 control procedures and 2304 intervention procedures. Distributions of patient and procedure characteristics across control and intervention arms are reported in Table 1. There were no differences between patients in age, sex, or comorbidity from control to intervention, though patients more often underwent orthopedic procedures, elective procedures, and regional anesthesia in the intervention arm.

TABLE 1

Patient and Procedure Characteristics of the Stepped Wedge Cluster RCT Study Sample (n = 3702) in a Norwegian University Hospital in 2009–2010

Characteristic Category	Control (n = 1398)	Intervention* (n = 2304)	P Value†
Age in years, mean (SD)	53.5 (23.4)	53.9 (23.4)	0.621
Male sex, n (%)	759 (54.3)	1247 (54.1)	0.919
Comorbidity by ASA, n (%)			0.107
ASA I	238 (17.0)	464 (20.1)
ASA II	568 (40.6)	964 (41.9)
ASA III	474 (33.9)	700 (30.4)
ASA IV	57 (4.1)	86 (3.7)
ASA V	2 (0.1)	2 (0.1)
No ASA score	59 (4.2)	87 (3.8)
Surgical procedure, n (%)			<0.001
Orthopedic	721 (51.6)	1557 (67.6)
Thoracic	293 (21.0)	392 (17.0)
Neuro	384 (27.5)	355 (15.4)
Surgery, n (%)			0.001
Elective	693 (49.6)	1274 (55.3)
Emergency	705 (50.4)	1030 (44.7)
Anesthesia, n (%)			<0.001
Regional	446 (32.9)	1013 (45.5)
General	909 (67.1)	1213 (54.5)

*Procedures that include full use of WHO SSC, partial use of WHO SSC, or noncompliance.

†From Pearson's exact χ2 test, except t test for age.

ASA indicates American Society of Anaesthesiologists’ risk score; RCT, randomized controlled trial; SD, standard deviation.

Implementation Outcomes (Fidelity of Checklist Usage)

We measured the fidelity to the use of each SSC part. In the intervention group there was complete compliance with 1 part of the SSC only (mostly Sign in or Time out), in 4.7% (109/2304) of the surgical procedures. Combinations of 2 parts (Sign in and Time out, Time out and Sign out, or Sign in and Sign out) being fully utilized were found in 8.5% (196/2304) of the procedures. Full compliance, using all 3 parts (Sign in, Time out, and Sign out) of the SSC, was identified in 75.7% (1743/2304) of the procedures. A total of 88.9% (2048/2304) had used any parts of the checklist, including all cases of complete compliance with 1, 2, or 3 parts. Noncompliance with the checklists was 11.1% (256/2304) in intervention arm procedures.

Care Processes

The results of comparing all care process metrics from controls to intervention procedures and in procedures with high fidelity of SSC usage are reported in Table 2. Measures for preoperative site marking, normothermia protection (prewarmed intravenous fluids, prewarmed blankets, forced air warming blankets), and antibiotics before incision were all significantly more often used in the intervention procedures compared with the controls. When adjusting for elective and emergency procedures, surgical case-mix, and type of anesthesia, the use of normothermia protecting measures and infection prophylactic antibiotics remained better applied in the checklist arm of the trial (Table 3).

TABLE 2

WHO SSC Impact on Care Process Metrics in the Stepped Wedge Cluster RCT (n = 3702) in a Norwegian University Hospital (2009–2010)

	Control	Intervention*		No Checklist Parts Used vs. Control		All SSC Parts Used vs. Control
Care Process Metrics Category	(n = 1398) Cases (%)	(n = 2304) Cases (%)	P Value†	(n = 256) Cases (%)	P Value†	(n = 1743) Cases (%)	P Value†
Site marking	971 (69.4)	1689 (73.3)	0.012	140 (54.7)	<0.001	1336 (76.6)	<0.001
Prewarmed intravenous fluid	766 (54.8)	1477 (64.1)	<0.001	136 (53.1)	0.633	1152 (66.1)	<0.001
Prewarmed regular blankets	1049 (75.0)	1856 (80.6)	<0.001	183 (71.5)	0.242	1439 (82.6)	<0.001
Forced air warming blankets	494 (35.3)	977 (42.4)	<0.001	58 (22.7)	<0.001	815 (46.8)	<0.001
Antibiotics			<0.001		<0.001		<0.001
Antibiotics before incision	762 (54.5)	1454 (63.1)		118 (46.1)		1194 (68.5)
Antibiotics after incision	174 (12.5)	228 (9.8)		85 (33.2)		143 (8.2)
No antibiotics	462 (33.0)	624 (27.1)		53 (20.7)		406 (23.3)

*Full use of WHO SSC, partial use of WHO SSC, and noncompliance.

†Pearson's exact χ2 test.

TABLE 3

WHO SSC Impact on Care Process Metrics in the Stepped Wedge Cluster RCT (n = 3702) in a Norwegian University Hospital (2009–2010)

	Intervention Procedures vs. Control			Use of All 3 WHO SSC Parts vs. Control
Care Process Metrics	OR	95% CI	P Value	OR	95% CI	P Value
Intravenous fluid (room tempered* vs. prewarmed)	1.46	(1.23, 1.73)	<0.001	1.53	(1.27, 1.85)	<0.001
Blankets (room tempered* vs. prewarmed)	1.31	(1.10, 1.56)	<0.001	1.44	(1.19, 1.75)	<0.001
Forced air warming (regular* vs. forced)	1.25	(1.07, 1.45)	<0.001	1.43	(1.22, 1.68)	<0.001
Antibiotics (no* vs. preoperative provided)	1.25	(1.07, 1.48)	<0.001	1.51	(1.27, 1,79)	<0.001
Site marking (no marking* vs. marking)	1.01	(0.82, 1.24)	0.966	1.23	(0.97, 1.55)	0.084

*Reference value.

OR indicates odds ratio; P value = from likelihood ratio test in logistic regression adjusted for emergency vs. elective surgery, surgical case-mix, and anesthesia provided.

Patient Outcomes

Primary endpoints are reported in Table 4. Complications including respiratory, cardiac, surgical infections, wound rupture, bleeding, and blood transfusions were all significantly reduced in the intervention arm of the trial. In procedures with no use of the checklist (n = 256), there was a borderline significant reduction for infections and wound rupture, but not for the remaining outcomes.

TABLE 4

WHO SSC Impact on Patient Outcome in the Stepped Wedge Cluster RCT (n = 3702) in a Norwegian University Hospital (2009–2010)

	Control	*Intervention		No Checklist Parts Used vs. Control		Used All Parts of the WHO SSC vs. Control
Main Complications	(n = 1398) Cases (%)	(n = 2304) Cases (%)	P Value	(n = 256) Cases (%)	P Value	(n = 1743) Cases (%)	P Value
Cardiac	112 (8.0)	116 (5.0)	<0.001	15 (5.9)	0.253	81 (4.6)	<0.001
Respiratory	116 (8.3)	93 (4.0)	<0.001	20 (7.8)	0.807	60 (3.4)	<0.001
Infection	104 (7.4)	82 (3.6)	<0.001	10 (3.9)	0.043	57 (3.3)	<0.001
Wound rupture	25 (1.8)	5 (0.2)	<0.001	0 (0.0)	0.044	5 (0.3)	<0.001
Bleeding†	36 (2.6)	24 (1.0)	<0.001	3 (1.2)	0.190	17 (1.0)	<0.001
Blood transfusions‡	95 (6.8)	123 (5.3)	0.072	19 (7.4)	0.788	78 (4.5)	0.005

*Intervention (include full use of WHO SSC, partial use of WHO SSC, and noncompliance).

†Bleeding: is postoperative bleedings as recorded from ICD-10 codes.

‡Blood transfusions: are transfusions provided intraoperatively during surgical procedures; P value indicates analysis using Pearson's exact χ2 test.

To statistically control for patient and procedure characteristics and process metric effects on complications, we used logistic regression analysis. Results are presented in Table 5. Use of forced air warming reduced odds ratio (OR) for cardiac complications and wound ruptures significantly. Further, infection prophylactic antibiotics provided before incision reduced OR for infections and wound rupture. In the intervention arm the SSC effects remained significant for all complications except respiratory complications, when adjusted for time effects (variation in process metrics and patient outcomes over time, i.e., per study month).

TABLE 5

Results From Logistic Regression Analyses of Complications on Checklist Fidelity in the Stepped Wedge Cluster Randomized Controlled Trial in a Norwegian University Hospital (2009–2010)

Complications
SSC Compliance (CA = Reference)	CA	IA	Cardiac			Respiratory			Infections			Wound Rupture*			Bleeding
	n	n	OR	95% CI	P Value	OR	95% CI	P Value	OR	95% CI	P Value	OR	95% CI	P Value	OR	95% CI	P Value
None used†	1398	256	0.72	(0.41, 1.25)	0.236	0.94	(0.58, 1.54)	0.795	0.51	(0.26, 0.98)	0.044	–	–	0.996	0.45	(0.14, 1.47)	0.185
1 part used†	1398	109	0.67	(0.29, 1.56)	0.351	0.53	(0.21, 1.33)	0.177	0.60	(0.24,1.50)	0.273	–	–	0.996	0.71	(0.17, 2.98)	0.637
2 parts used†	1398	196	0.88	(0.50, 1.57)	0.673	0.47	(0.23, 0.98)	0.044	0.67	(0.34, 1.30)	0.237	–	–	0.995	0.39	(0.09, 1.63)	0.197
3 parts used†	1398	1743	0.56	(0.42, 0.75)	<0.001	0.39	(0.29, 0.54)	<0.001	0.42	(0.30, 0.59)	<0.001	0.16	(0.06, 0.41)	<0.001	0.37	(0.21, 0.67)	0.001
Any parts used†	1398	2048	0.60	(0.45, 0.79)	<0.001	0.41	(0.30, 0.55)	<0.001	0.45	(0.33, 0.62)	<0.001	0.13	(0.05, 0.35)	<0.001	0.39	(0.23, 0.67)	0.001
All cases†	1398	2304	0.61	(0.47, 0.80)	<0.001	0.47	(0.35, 0.62)	<0.001	0.45	(0.33, 0.62)	<0.001	0.12	(0.05, 0.31)	<0.001	0.40	(0.24, 0.67)	0.001
Intervention‡	1398	2304	0.61	(0.44, 0.85)	0.003	0.98	(0.55, 1.76)	0.051	0.52	(0.38, 0.71)	<0.001	0.14	(0.05, 0.34)	<0.001	0.55	(0.32, 0.96)	0.035

P values in the regression models are based on the likelihood ration test.

*For the variable “Wound rupture” there were too few cases to calculate OR and 95% CI for None used, 1 part used, and to 2 parts used.

†Fidelity of “SSC parts used” entered into the logistic regression model I (3 parts used = full checklist compliance).

‡SSC effects adjusted for age, sex, case-mix, comorbidity, anesthesia type, knife time, study time points, and process metrics in the logistic regression model II's final step.

CA indicates control arm; IA, intervention arm; OR, odds ratio.

Postoperative bleeding identified through ICD-10 codes decreased from 2.6% (36/1398) to 1.0% (24/2304) in the intervention arm (P < 0.001). In support to this finding, adjusted for patient and procedure characteristics the risk of postoperative bleeding was reduced in the intervention steps (Table 5). Further, evaluating intraoperative blood loss percentiles, there was significant reduction of 750 mL to 1000 mL blood loss (6.0% vs. 4.5%), and increase for no (0–49 mL) or minor bleeding (50–249 mL)—25.2% vs. 28.6% and 21.1% vs. 24.3%, respectively (P = 0.006) (SDC 3). The need of blood transfusion also decreased in the procedures where the SSC had been applied (Table 4). Distribution of blood transfusions with plasma, erythrocytes, and platelets is presented in Figure 2.

FIGURE 2

WHO SSC Impact on Intraoperative Blood Transfusions—in the Stepped Wedge Cluster RCT, Haukeland University Hospital (2009–2010). All blood transfusions = 1 or more transfusions per surgical procedure.

Adjusted for patient and procedure characteristics and care process metrics, the risk of having a blood transfusion was reduced when using all 3 parts of the SSC, with OR 0.63 (95% CI, 0.43–0.91). OR was 5.81 (95% CI, 3.34–10.01) in emergency procedures; 1.94 (95% CI, 1.16–3.27) in general anesthesia; 3.07 (95% CI, 2.31–4.01) by increasing ASA classification; 1.01 (95% CI, 1.01–1.02) by increasing knife time (minutes); 2.68 (95% CI, 1.26–5.69) in orthopedic procedures; and 0.40 (95% CI, 0.20–0.81) for neurosurgical procedures. Forced air warming blankets were more frequently used in procedures requiring blood transfusions OR 2.68 (95% CI, 1.26 to 5.69).

Costs for blood transfusion units in USD were overall recorded per procedure for all transfusion units of plasma, erythrocytes, or platelets administered to patients. Mean blood transfusion costs in control procedures were USD 46.42 vs. USD 36.39 in the intervention procedures (P = 0.092). The cost was USD 28.03 in intervention procedures utilizing the SSC with high fidelity (all 3 parts, P = 0.007), representing a 40% cost reduction of blood transfusions.

DISCUSSION

We studied in detail how the quality of the SSC implementation impacts its clinical effectiveness. Our results indicate that better use of the checklist (ie, high-fidelity application) is needed for clinical effectiveness to materialize. Both process metrics and patient outcomes improved when all parts of the checklist were utilized. In line with the UK study on the SSC,[10] our results show that high-fidelity use of the checklist, including all 3 parts of the checklist, provides the lowest rates of odds ratio (Table 5).

Good-quality implementation of the SSC improved both care processes and outcome for patients. The findings correspond well to the clinical improvement model that we hypothesized in Figure 1. The outcome improves as a function of better care processes being in place and due to good actual use of the SSC.

Our results replicate early findings by Haynes et al—the SSC improved safety and process measures (airway evaluation, pulse oximeter use, intravenous catheter, antibiotics, patient identity and site marking, and sponge count), though their process measures were not compared directly to patient outcomes.[11] The WHO recommends monitoring safety and care processes associated with the SSC implementation.[16] This is in accordance with Donabedian's framework for improvement that outlines care structures, processes, and outcomes.[17,23] The strength of this perspective lies within this interrelationship where structure (the SSC in this case) improves the process, and both structure and process then improve outcomes.[16,17] This was especially evident in the use of hypothermia preventing care processes (forced air warming) and timeliness of infection prophylactic antibiotic provided in the operating room.

Even mild hypothermia (34°C to 36°C) is known to increase the incidence of surgical wound infections,[24] blood transfusions,[25] prolonged hospitalization,[24,25] and prolonged recovery from drugs.[26] Hence, to obtain patients’ normothermia is of vital importance to prevent intra- and postoperative complications. Ensuring normothermia may be associated with increased use of prewarmed blankets and forced warming air blankets after the SSC implementation (Table 2). Both the use of the SSC and active warming blankets with forced air were significantly related to lower risk of surgical wound rupture and cardiac complications. These results correspond to previous research that indicated a 55% reduction in risk of morbid cardiac events when normothermia was maintained.[27] Hypothermia is well known to increase risk of cardiac complications due to elevations in blood pressure, heart rate, plasma concentrations of catecholamine, and thus myocardial ischemia by turning myocardial oxygen balance into a net deficit.[28] With an increased use of prewarmed intravenous fluid, prewarmed blankets, and forced warming air that correspond to items on the SSC, we find it reasonable to attribute the effect on surgical wound ruptures and cardiac complication to the checklist intervention and improved hypothermia preventing care processes.

Another major finding is the improved timeliness of prophylactic antibiotics provided in operating rooms through good use of the SSC. Antibiotics were administered to patients significantly more frequent before incision and fewer times after incision in the intervention procedures. Our results underline the recommendations on preoperative measures for surgical site infections recently released by the WHO Guideline Development Group. Surgical antibiotic prophylaxis is to be administered within 120 minutes before incision customized to the half-life time of the antibiotics.[29] Optimal timing of antibiotics has been estimated to potential reduce infections in cardiac surgery by 9% to 31%.[30] We identified a significant reduced odds ratio for having a surgical infection, 0.54 (95% CI, 0.37–0.79), when antibiotics were provided before incision rather than no antibiotics given or antibiotics provided after incision. The use of checklists seems to influence on better timing of antibiotics and reduction of surgical infections. The efficacy of antibiotic prophylaxis in preventing surgical site infections has been clearly established,[31] hence antibiotic items on the checklist may optimize and ensure adequate tissue levels of the antibiotic microbial prophylaxis according to the half-life time of the drug at the initial incision.

In a recent randomized controlled trial of a modified surgical safety checklist, surgical wound, abdominal and bleeding-related complications were significantly lowered in the checklist arm of the study.[32] Similarly, we observed a significant reduction in postoperative bleeding from 2.6% to 1.0% and significant improvement of intraoperative bleeding in the SSC intervention procedures. Adding to this, we found a significant reduction in transfusions of plasma, erythrocytes, and platelets in the SSC intervention procedures. The clinical relations between the checklist, intraoperative bleeding, and need of blood transfusion are multifactorial; however, we find the 2 hypothermia preventing items on the checklist to be important. These relations are supported by the improvement seen in use of forced air warming (Tables 2 and 3) and subsequent reductions in bleedings and blood transfusions. A plausible explanation is prevention of hypothermia induced by the checklist intervention.[25]

Implementation of the SSC in US hospitals was estimated to generate cost savings once it prevents at least 5 major complications in hospitals with a 3% baseline rate on major postoperative complications.[33] We observed an approximate 40% cost reduction associated with blood transfusions after implementation of the SSC in our Norwegian hospitals. This result suggests a potential economic benefit of the SSC intervention with improved care processes and patient outcomes.

Strengths and Limitations

The use of a stepped wedge cluster randomized controlled methodology has been described as a robust study design for quality improvement clinical trials.[9] It prevents extraneous influences as it has controls and intervention steps across the same time periods, and offers the possibility for modeling the effects of time on the effectiveness of the SSC intervention.[19,22] However, our study has some limitations. Routinely collected data may be hampered by random errors or inaccuracy regarding data quality. In our study, data on SSC compliance were prospectively recorded on paper forms. These data were validated against concurrent electronic registrations of checklist utilization.[12] Use of routine data may also have been of some benefit, as it made it possible to leave the healthcare personnel involved unaware of the specific data of interest to the study. This also applied to process data, as well as outcome measures. In our study we did not have access to care process metrics associated with every single item of the SSC, which is a limitation of our study. Items that did not have corresponding metrics could also have improved the care processes and may have contributed further to improvement of the outcomes. There were no changes in how routine data were recorded in the study period. Random errors would most likely be equally present both before and after the intervention steps.

Intraoperative bleeding was significantly lower in procedures where the SSC had been utilized. The size of this reduction does perhaps not seem clinically relevant when presented as average group values, and might need further exploration. However, the finding was strengthened by a significant reduction of blood transfusions in the SSC procedures. Another possible limitation was that the process metric “forced air warming” increased the odds ratio for having a blood transfusion. Initially, this might seem contradictory, but preventing hypothermia to prevent further blood loss, might render forced air warming more frequently used in patients with large bleedings.[24,25] Thus, this offer a clinical explanatory mechanism to the seemingly increased likelihood of bleeding by “forced air warming.”

Another limitation was lack of patients’ core temperature as a parameter. However, due to incomplete data as temperature measures for all surgical procedures at the time of the study, and to avoid introducing competing interventions, we omitted use of patients’ core temperature as process metric. Further, for other important items like the team briefing and different risk assessments there were no available metrics. This might represent a limitation for our study as these items also may have contributed to the improved outcomes, however difficult to measure.

Between control and intervention steps there were no differences in patient characteristics. However, we acquired a larger proportion of orthopedic procedures and regional anesthesia in the intervention part of the study, due to the stepped wedge design, as following random allocation the intervention started in orthopedic surgery (with largest number of procedures). Variation in elective and emergency procedures may have been influenced by the intervention itself, as we previously reported a drop in unplanned returns to the operating room from 1.7% to 0.6%, P < 0.001.[12] To control for these indifferences from control to intervention procedures we used logistic regression analysis to adjust for case mix and possible confounding effects. In surgical quality service improvement trials it is difficult to control for complexity and all possible factors that may influence or explain outcome.

Future Research

Our study sheds some light in what may be defined as clinical “micro-processes” within the operating room. The need remains to better understand how the complexity in hospital organization, safety culture, team cohesion, and communication impact on how well surgical improvement interventions are introduced and implemented, and how in turn care processes and patient outcomes improve as a result.[34] Further studies are necessary to establish quantitative relationships between specific checklist items and related care processes and complications.

CONCLUSION

This study successfully applied Donabedian's improvement framework of clinical structures, processes, and outcomes as a clinical causal model for the SSC intervention. Use of SSC improved operating room care processes; subsequently, high-quality SSC implementation and improved care processes led to better patient outcomes.