Validation of a Norwegian version of SURgical PAtient Safety System (SURPASS) in combination with the World Health Organizations' Surgical Safety Checklist (WHO SSC).

Introduction: Surgical safety checklists may contribute to reduction of complications and mortality. The WHO's Surgical Safety Checklist (WHO SSC) could prevent incidents in operating theatres, but errors also occur before and after surgery. The SURgical PAtient Safety System (SURPASS) is designed to intercept errors with use of checklists throughout the surgical pathway. Objective: We aimed to validate a Norwegian version of the SURPASS' preoperative and postoperative checklists for use in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC. Methods and materials: The validation of the SURPASS checklists content followed WHOs recommended guidelines. The process consisted of six steps: forward translation; testing the content; focus groups; expert panels; back translation; and approval of the final version. Qualitative content analysis was used to identify codes and categories for adaption of the SURPASS checklist items throughout Norwegian surgical care. Content validity index (CVI) was used by expert panels to score the relevance of each checklist item. The study was carried out in a neurosurgical ward in a large tertiary teaching hospital in Norway.
Results: Testing the preoperative and postoperative SURPASS checklists was performed in 29 neurosurgical procedures. This involved all professional groups in the entire surgical patient care pathway. Eight clinical focus groups revealed two main categories: 'Adapt the wording to fit clinical practice' and 'The checklist items challenge existing workflow'. Interprofessional scoring of the content validity of the checklists reached >80% for all the SURPASS checklists. Conclusions: The first version of the SURPASS checklists combined with the WHO SSC was validated for use in Norwegian surgical care with face validity confirmed and CVI >0.80%. Trial registration number: NCT01872195.

Introduction

Surgical complications are a global concern. A review of closed healthcare claim cases including complications showed that it would be possible to prevent 50% of the cases.1 A common problem which is known to complications is poor communication.2 Tools such as safety checklists have been introduced to enhance teamwork, communication and reduce patient safety risks.3 Use of checklists has been shown to reduce surgical complications and mortality.4–6 WHO’s Surgical Safety Checklist (WHO SSC) was introduced in the operating theatres (OTs) in two Norwegian hospitals in 2009–2010.6 However, the in-hospital surgical pathway is comprehensive and consists of multidisciplinary involvement and interactions in OTs and in the admission phase, preoperative phase, postanaesthesia care unit (PACU) and postoperative ward care.7 Transfers through different departments with loss of information throughout the clinical pathway may be a threat to patient safety.8 Complications are known to occur also in the preoperative and postoperative phases of surgery.9 Many risk factors have been described, such as failing to identify allergies,10 lack of antibiotic prescriptions11 and follow-up on venous thromboembolism risk and prophylaxis.12 To our knowledge, there is only one validated checklist concept that systematically cover the total surgical pathway with personal checklists for the involved key personnel used through all critical transfer points in the care process: the Dutch SURgical PAtient Safety System (SURPASS) checklists.13

The SURPASS consists of 11 checklists covering the total surgical flow, from admission to discharge. Introduction of the SURPASS checklists in six Dutch hospitals reduced complications from 27.3 (95% CI 25.9 to 28.7) to 16.7 (95% CI 15.6 to 17.9). The mortality was reduced from 1.5% (95% CI 1.2 to 2.0) to 0.8% (95% CI 0.6 to 1.1).5 The WHO SSC has been implemented in all hospitals in Norway as part of the Norwegian patient safety programme ‘In Safe Hands’.14 Due to mandatory use of the WHO SSC, it was not possible to introduce all parts of the more comprehensive SURPASS system. Nevertheless, it seemed to be feasible to introduce the preoperative and postoperative SURPASS checklists in combination with the WHO SSC in clinical practice. Thus, this needed further investigation. We aimed to translate the SURPASS’ five preoperative and three postoperative checklists and validate the SURPASS version in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC for use in Norwegian surgical care.

Methods and materials

Translation and validation of the SURPASS checklists content into Norwegian flow of surgical care followed the WHO guidelines,15 recommended for translation and adaption of instruments. The process consisted of six steps: (1) forward translation; (2) testing the content; (3) focus groups; (4) expert panel; (5) back translation and (6) approval of the final version.

The study was carried out in a neurosurgical unit in a large tertiary teaching hospital in Norway, referral for 1.1 million inhabitants, performing all common neurosurgical procedures both in children and adults.

WHO Surgical Safety ChecklistSSC

The established WHO SSC consists of three checklists to be performed within the OT at three definite moments in surgery: before induction of anaesthesia, before incision and at the end of surgery.16 The checklist was in 2009 translated to Norwegian17 by clinical experts including surgeons, anaesthesiologists, nurse anaesthetists, OT nurses and quality improvement officers. It was back translated to English by native English-speaking personnel and became the official Norwegian version.18 The WHO SSC was implemented in five surgical departments, including neurosurgery. Effects of using the checklists have been validated through previous published work.6 17 19 Further implementation of the WHO SSC at the remaining surgical departments followed WHO’s implementation guide with adaptation to local use.20 21

The SURPASS checklists

The SURPASS checklists consist of five preoperative, three intraoperative and three postoperative checklists. The preoperative and postoperative checklists are individualised to fit the healthcare providers’ professional responsibility. The original version of the SURPASS checklists13 was developed in three steps: (1) literature studies on human processes and adverse events after surgical procedures, (2) observations of safety risk events in clinical practice throughout the perioperative care and (3) practical and effectiveness evaluation of the checklists. The content was validated by observing safety deviations in clinical practice in comparison with checklist items.13 This process was to ensure that practice and theory corresponded. The original preoperative and postoperative phases of SURPASS consisted of 63 checklist items. In addition, two items on the preoperative checklist for surgeons were to be used in case of local anaesthesia without anaesthesiologist.

In contrary to the WHO SSC, which are performed by the surgical team, the preoperative and postoperative SURPASS checklists are personalised and completed by individual health professionals in charge of specific care details through the surgical care pathway. We chose to add specifically the preoperative and postoperative parts of the SURPASS checklists to the already established intraoperative WHO SSC in our hospital and combine them in one comprehensive perioperative checklist.

Forward translation

An English translation of the content was provided from the SURPASS copyright holders5 in addition to the original Dutch version. Translation of the checklist content into Norwegian was first carried out by professional translators (Semantix AS, Stavanger, Norway). Then, the translated and the English versions of the checklists were reviewed by three clinical experienced researchers (AS, ASH and ES). Cross-cultural adaptation of surgical workflow and logistics in checkpoints from Dutch to Norwegian standards were ensured in close collaboration with surgeons and healthcare personnel from the neurosurgical department testing the checklists. This also investigated the face validity and feasibility. Three items were left out from the original Dutch preoperative ward nurse checklist due to lack of local existing protocols and procedures at the time of investigations: screenings for decubitus; risk of patient falls; and delirium. All three screening protocols were under development and scheduled to be introduced at a later stage. One item for the discharging nurse concerning home regimen explained to patient was left out due to being covered in standard discharging procedures. Two new procedures were implemented that contribute to two new checklist items on the preoperative ward nurse checklist: body temperature controlled 1 hour before entrance to the OT (not in the original version) and patient identification tags on both wrists (in the original version: name tags and barcode on both wrists). One of the original checklists assigned to an anaesthesiologist or intensivist when transferring the patient from PACU or intensive care unit to hospital wards was changed and assigned to the PACU nurse.

Testing the content

Before testing the checklists, all groups of healthcare professionals received at least one educational session. The personnel involved in neurosurgery were ward doctors (neurosurgical resident/consultant in neurosurgery/final year student resident), ward nurses (registered nurses (RNs)), neurosurgeons, anaesthesiologists, OT nurses (RNs with graduate certificate in operating room processes), PACU nurses (RNs or graduate certificate in intensive care) and discharging doctors (neurosurgical resident/consultant in neurosurgery/final year student resident) and nurses (ward nurse and RNs). All personnel involved received information by email and informative posters that were displayed in the department. Training followed the principles of Conley and colleagues,22 by explaining why the checklists were tested and showing how to use the different checklists. The implementation team consisted of key clinical personnel, the research group and the middle level of management for the involved groups. Paper version checklists were used individually by personnel at each preparatory step of the surgical pathway. All the checklists had user instructions attached. The lists were designed to check whether all necessary procedures had been completed, hence different from a to-do list.23 During the test period, it was mandatory to use the preoperative parts of the checklists. In agreement with the department head, consequences of not completing the checklists resulted in delayed surgery.

During the test period, the checklist users were asked to write feedback notes on a daily basis regarding wording of the checklist items. This was to determine whether the wording was precise and to get an understanding of optimal time-points for completion of the checklists.

The implementation team was available to clarify doubts and follow-ups throughout the test period. All the surgeons were asked individually on their experiences of using the preoperative and discharging checklists.

Focus groups

After testing the checklists in clinical settings, we needed more systematic information regarding the checklist users’ perspective on usage and existing workflow in relation to checklist compliance.24 Eight focus groups were carried out by two moderators. We planned to perform interviews in small focus groups (two to five participants) with a strategic sample of healthcare professionals. Respondents being potential users of the SURPASS checklists, including surgeons, anesthesiologists, ward doctors, ward nurses, OT nurses and PACU nurses with mixed length of experiences, were selected. The interviews were scheduled to last up to 60 min. Trained interviewers and moderators (AS, HVW, ASH and ES) conducted the focus group interviews. The interviews were carried out in hospital settings close to the wards and OTs to minimise use of time away from clinical work. The checklist items formed the interview guide. Data from the interviews were noted as condensed meaning units on a paper form. The participants reported their clinical experience, sex and profession. We used qualitative content analysis to identify codes and categories to assess the items adaption to the existing work flow.25

Expert panels

Each item on the checklists were subsequently tested by expert panels for all the eight new SURPASS checklists using the content validity index (CVI).26 To score the CVI, we used eight panels with experts. The experts were instructed to score the content from a general surgical angle—covering all the surgical areas, not merely neurosurgery. The CVI scoring was performed to test relevance and comprehensiveness of precise and clear wording of the checkpoints.27 The experts rated each checkpoint item on a four-point scale: 1=not relevant, 2=somewhat relevant, 3=quite relevant and 4=highly relevant.28 Item content validity scores (I-CVI) were used to guide revision of wording or questions of deleting items or text. To reveal the total content validity score of the checklist or scale (S-CVI), the proportion of experts who have scored 3 or 4 were calculated.26

Back-translation and final approval of the SURPASS checklists

Following a forward translation, testing of the content in clinical practice, focus groups and validation by expert panels, the checklists were back-translated from Norwegian to Dutch by a native Dutch speaker. The back-translated checklists, including both the SURPASS parts and the WHO SSC were presented to the Dutch SURPASS copyright holder for approval.

Results

The content of the original SURPASS checklists has previously been published.5 After forward translation of the checklist content, managers and the different clinical professionals ensured that the different checklist contents were assigned to the responsible healthcare professional following Norwegian standards and legislation. The item ‘obtaining written consent’ is not required by Norwegian legislation; thus, this checklist item was left out. Adjustments and cross-cultural adaptations to local workflows needed to be performed: for example, ward doctors in the Netherlands are to check on: relevant imaging present; in Norway, the surgeons assess the images and the OT nurses check for the presence of the images in the OT. Also for Dutch ward doctors: relevant laboratory checks, including cross-typing; in Norway, ward nurses check for cross-typing, while the surgeons and anaesthesiologists control the laboratory results. All healthcare professional groups engaged in neurosurgery each confirmed face validity and feasibility of their respective checklist items before the checklists were tested in clinical practice.

We tested the checklists in 29 neurosurgical procedures performed over 3 weeks in June and July 2012. In each surgical procedure, 11 checklists were used, which includes: the five new preoperative SURPASS checklists, the established three parts of WHO SSC and the three new postoperative SURPASS checklists. All the healthcare professional groups engaged in neurosurgery were represented. Compliance rates to the different checklists are presented in figure 1. The SURPASS checklists used here included 64 checklist items, in addition to two items on the preoperative checklist for surgeons to be used in case of local anaesthesia without an anaesthesiologist involved. The test revealed that some items had to be moved to other professional groups due to differences in national and local work assignments and work flow, and some items needed to be reformulated for clarity, specificity and simplicity.

Figure 1

Compliance to the preoperative and postoperative SURPASS checklists according to professional background when testing the content in 29 neurosurgical procedures, June–July 2012, in one Norwegian hospital. PACU, postanaesthesia care unit.

The focus groups involved professionals having been assigned the five preoperative and three postoperative SURPASS checklists, with 2–5 professionals in each group. All the interviews, except one, had both an interviewer and a moderator. Two interviews had one healthcare provider involved, all together 25 different professionals participated. The participants had a wide range of working experience, from 6 months to 35 years, with 52% being females. Three identified codes ‘change of wording’; ‘responsibility’ and ‘organisation (of when to do the checklist)’ constituted the main categories of ‘Adapt the wording to fit clinical practice’ and ‘The checklist items challenge existing workflow’ (figure 2).

Figure 2

Qualitative content analyses to understand eight focus groups’ perspectives on the tested preoperative and postoperative SURPASS checklist content for neurosurgical procedures in one Norwegian hospital. OT, operating theatre; PACU, postanaesthesia care unit.

Following careful text adjustments after testing the checklists in clinical practice, and adjusting items according to the suggestion from focus groups, the next step in the validation process was the CVI scoring. The expert panels’ characteristics are shown in table 1.

Table 1

Characteristics of neurosurgical personnel scoring content validity index (CVI) of the preoperative and postoperative SURPASS checklists after testing, focus groups and adjustments according to feedback in the SURPASS validation study in a tertiary teaching hospital, in Norway, 2012

Profession (n)	Sex, female/male	Age, mean years (range)	Worked in the profession, mean years	Worked as a junior, mean years	Worked as a specialist, mean years
Operating theatre nurse (5)	5/0	56 (48–61)	26	–	19
Ward doctor (6)	3/3	33.8 (29–39)	6.8	3.5	–
Surgeon (6)	0/6	48 (31–62)	20.3	3 (n=2)	24 (n=4)
Anaesthesiologist (6)	1/5	42 (31–64)	14	2 (n=1)	13.8 (n=5)
Ward nurse (6)	5/1	31.5 (26–39)	8.3	8.1	-
PACU nurse (6)	4/2	39.3 (33–54)	15.1	–	6.4
Discharging doctor (6)	0/6	48 (31–62)	20.3	–	15.6
Discharging nurse (6)	5/1	31.5 (26–39)	8.3	–	8.1

PACU, postanaesthesia care unit; SURPASS, SURgical PAtient Safety System.

Altogether 35 different healthcare personnel scored CVIs. Six surgeons and six ward nurses scored on both the preoperative and discharging checklist. The scorings on I-CVI and S-CVI are represented in table 2.

Table 2

The item content validity index (I-CVI) and scale content validity index (S-CVI) scores by the neurosurgical experts evaluating the preoperative and postoperative SURPASS checklists after testing, focus groups and adjustments according to feedback in the SURPASS validation study in a tertiary teaching hospital, in Norway, 2012

Experts (n)	Checklist items rated	Items rated 1 or 2*	Items rated 3 or 4†	Calculating the mean I-CVI	S-CVI
Operating theatre nurse (5)	5	0	25	25/25	1.00
Ward doctor (6)	5	3	27	27/30	0.90
Surgeon (6)	9	9	45	45/54	0.83
Anaesthesiologist (6)	7	4	38	38/42	0.90
Ward nurse (6)	13	11	67	67/78	0.86
PACU nurse (6)	6	1	35	35/36	0.97
Discharge doctor (6)	10	10	50	50/60	0.83
Discharge nurse (6)	5	4	26	26/30	0.87

*1=not relevant; 2=somewhat relevant.

†3=quite relevant; 4=highly relevant.

PACU, postanaesthesia care unit; SURPASS, SURgical PAtient Safety System.

Examples of items having a low score (1 and 2): for surgeons: preoperative marking of the incision site; and preoperative hair removal. For ward nurses: marking of the incision site.

Back translation of the Norwegian validated version

Following careful adjustments after validation, the Norwegian version of the preoperative and postoperative parts of the SURPASS checklists finally consisted of 60 checklist items distributed on five preoperative and three postoperative checklists. In addition, one item was to be performed preoperatively by surgeons in case of local anaesthesia without an anaesthesiologist involved. All the original checklist items excluding the three ward nursing screenings and obtained consent were included in the Norwegian version. The content of the tested checklists and the corresponding content having been back translated are shown in online supplementary digital content 1. The back-translated checklists, including both the SURPASS parts and the WHO SSC, were approved by the Dutch SURPASS copyright holder.

Discussion

The English version of SURPASS’ five preoperative and three postoperative checklists were validated together with the established three parts of WHO SSC in a neurosurgical department in a tertiary hospital in Norway. The validation process consisted of six steps, including forward translation, testing the content, focus groups, expert panels, back translation and approval of the final version. There was a general positive attitude towards using checklists, although critique, reluctance and questions regarding the checklists themselves and on safety-effects were also raised. Checklist scepticism has also been documented for years in other healthcare settings.22 29–32

Before testing the content and the flow of checklists, there was a close collaboration with management and health personnel within each profession for all checklist parts. The Dutch and Norwegian standards of healthcare are very similar, but some differences in healthcare providers’ responsibilities were disclosed. To overcome this, some items were assigned to other professions’ checklists. From the literature and our previous experience on implementation of the WHO SSC, we observe that including key stakeholders at an early stage for buy-in and to increase ownership in the process is recommended.33–35 Face validity and feasibility were confirmed before testing the content in clinical practice.

Testing the checklists in clinical practice revealed that there were still challenges concerning wording and the existing workflow. Several studies have identified that change of workflow following checklist implementation may represent a barrier to engage the healthcare providers.35–38 Although many of the clinicians found a paper checklist most convenient for testing the content, there were logistic challenges that resulted in low compliance rates for the OT nurses. Some of the personnel were enthusiastic about systematically having a last check-up before transferring the patient. Some were engaged to give the test period a fair chance to succeed and were open-minded. Others were open on concerns, that is, another thing to spend time on in an already time-constraint environment. The managers were engaged and pointed out dedicated staff to follow up the test period. The implementation team involved and engaged the personnel thoroughly, on both group and individual levels and monitored the process closely. The WHO SSC was implemented in this hospital in 2009. It is mandatory to use, and it has a good compliance rate. However, discussions on issues regarding the WHO SSC were important, but the main focus was on testing the new SURPASS checklists.

To get a further insight into the challenges with the existing workflow and identify wording to be improved, we conducted focus group interviews. The focus groups had several suggestions for rephrasing list contents to adapt the wording and item content into clinical practice and workflow.

All the expert panels were instructed to score the CVI from a general surgical perspective. Still, the ‘low relevance’ scorings of specific checklist items were explained as not being important for the expert panel’s surgical discipline. However, these items could be judged as highly relevant checkpoints for other surgical departments and should be tailored to these settings accordingly. Thus, despite a low score, these items were not removed from the checklists being back translated due to generalisation to other specialities. However, the items were removed from neurosurgery checklists as a local adjustment. All the eight checklist scores had a CVI >0.80. A 90% agreement on CVI is regarded satisfactory with some authors,27 while others urge to have total agreement by all the experts if five or fewer experts.39 However, if six or more experts are scoring, the I-CVI is regarded as valid when 80% reach agreement.39 40 All the checklists reached an acceptable CVI score.

We recommend local adaptation and testing the content in new settings to disclose and terminate barriers before implementation of additional surgical checklists.

Strengths and limitations

A strength of this study is the inclusion of interprofessional key stakeholders in the early process of adjusting the content to Norwegian work assignments and flows. Another is the continuous process of testing the checklists in practice with all health professional groups represented. Generally, the similarities between Dutch and Norwegian surgical safety standards increased likelihood that the checklist contents followed existing workflow and procedures. Still, three items were not included on this checklist version due to lack of protocols and work processes corresponding to these items. All new protocols and work processes should of course be implemented properly before the checklists are introduced. Prior to checklist implementation, a thorough evaluation of context, assessing corresponding work processes and procedures to checklist items has also been recommended in the literature.35

It may be a possible limitation that the Norwegian version of the SURPASS checklists was validated in one department only. However, the original SURPASS checklists was developed through a great variety of surgical procedures and settings, to make adaptation of the checklists to other hospital departments feasible.13 Use of highly experienced and expert personnel when testing the checklists may be seen as a strength. Advices as to adaption and tailoring the content to the setting were followed.21

Conclusion

The SURPASS’ preoperative and postoperative checklists were successfully validated for use in Norwegian surgical care with high face validity and content validity (CVI >80%) and in combination with the WHO operative checklist. Adding new checklists in combination with the already established Sign In, Time Out and Sign Out parts of the WHO SSC was feasible in neurosurgery.