Development of the Surgical Patient safety Observation Tool (SPOT).

BACKGROUND: A Surgical Patient safety Observation Tool (SPOT) was developed and tested in a multicentre observational pilot study. The tool enables monitoring and benchmarking perioperative safety performance across departments and hospitals, covering international patient safety goals.
METHODS: Nineteen perioperative patient safety observation topics were selected from Dutch perioperative patient safety guidelines, which also cover international patient safety goals. All items that measured these selected topics were then extracted from available local observation checklists of the participating hospitals. Experts individually prioritized the best measurement items per topic in an initial written Delphi round. The second (face to face) Delphi round resulted in consensus on the content of SPOT, after which the measurable elements (MEs) per topic were defined. Finally, the tool was piloted in eight hospitals for measurability, applicability, improvement potential, discriminatory capacity and feasibility.
RESULTS: The pilot test showed good measurability for all 19 patient safety topics (range of 8-291 MEs among topics), with good applicability (median 97 (range 11·8-100) per cent). The overall improvement potential appeared to be good (median 89 (range 72·5-100) per cent), and at topic level the tool showed good discriminatory capacity (variation 27·5 per cent, range in compliance 72·5-100 per cent). Overall scores showed relatively little variation between the participating hospitals (variation 13 per cent, range in compliance 83-96 per cent). All eight auditors considered SPOT a straightforward and easy-to-use tracer tool.
CONCLUSION: A comprehensive tool to measure safety of care was developed and validated using a systematic, stepwise method, enabling hospitals to monitor, benchmark and improve perioperative safety performance.

Introduction

Improving patient safety remains a major public health concern1. Almost two‐thirds of in‐hospital adverse events are associated with perioperative care2. Improvement in perioperative patient safety can be achieved by observing and measuring safety performance, and by using the results to improve the perioperative process3, 4, 5 in a plan–do–check–act cycle. Benchmarking of perioperative safety performance may be used to compare departments, hospitals and methods, to improve patient safety.

National audits by the Dutch Health Care Inspectorate (Inspectie voor de Gezondheidszorg; IGZ) in 2007–2009 showed that perioperative care in the Netherlands could be improved by information transfer, clinical documentation, teamwork and coordination6, 7, 8. In response to the recommendations of the IGZ, national evidence‐based perioperative safety guidelines9, 10, 11 were developed between 2010 and 2012, applicable to both surgical and non‐surgical interventions in hospitals. Adherence to evidence‐based guidelines is associated with safer perioperative care and improved outcome12, 13, 14.

In 2012, the IGZ reaudited perioperative care and concluded that, although the perioperative process was better structured, there was still room for improvement. Furthermore, safety performance still varied considerably between hospitals. For example, although preoperative time‐out and postoperative sign‐out procedures were performed in all hospitals, approximately half of these procedures were incomplete. In addition, the IGZ announced a zero tolerance policy in cases of wrong‐side and wrong‐site surgery15.

There is a growing need for an observation tool that enables monitoring of guideline compliance, follow‐up and comparison between different specialties and hospitals, and identification of best practices. Until now, such a condensed and easy‐to‐use observation tool, developed and used by professional caregivers themselves, has not been available. The tool under development was designed to cover all 19 patient safety observation topics used by the IGZ for auditing Dutch hospitals (IGZ framework 2012) (Table 1)15. IGZ and guideline developers agreed that these topics represent the main safety observation topics. The topics are also in line with the six international patient safety goals highlighted by the Joint Commission International (JCI)16 as areas of concern in healthcare (Table 2).

Table 1

Perioperative patient safety observation criteria used by the Dutch Health Care Inspectorate15

Criterion	Topic	Details
1	Patient identification	Identification of the patient should be done using at least two out of three characteristics (patient name, date of birth, patient identification number). Identification is based on three independent sources: the (wakened) patient or legal representative, the patient's medical dossier and identification bracelet(s)
2	Handover from wards	It is strongly recommended to use structured checklists for handover of patients during the entire perioperative healthcare process
3	Pre‐time‐out before the operation	A pre‐time‐out is performed as an additional check when invasive preparations (e.g. regional anaesthetic block) for the operation are done in the preanaesthesia care unit of the OR. The attending anaesthetist together with a second attending person checks the following items: patient identification; type of operation; side and/or location of operation; possible allergies of the patient; the actual parameters of (anti)coagulation; and presence of appropriate materials. If possible, these checks are done with involvement of the conscious patient. The anaesthetist is responsible for performing and recording this safety verification hold
4	Time‐out before the operation	A time‐out is performed in the presence of the patient before induction of anaesthesia. The attending anaesthetist, surgeon and OR staff check the following items: correct patient; correct type of operation; correct side and/or location of the operation; the actual parameters of (anti)coagulation; necessity for administration of antibiotics; possible allergies of the patient; co‐morbidity of the patient; positioning of the patient during the operation; presence of appropriate personnel and materials; and other relevant details. The surgeon is responsible for performing and recording this safety verification hold
5	Intraoperative team communication	Communication between attending surgeon and anaesthetic team member(s) present is obligatory at the start and finish of the surgical procedure and at all points that are of interest to the condition or safety of the patient, or that will interfere largely with the activity of the other attending specialty
6	Handover at shift change during the operation	A handover is performed at all shift changes during the operation
7	Sign‐out after the operation	Under responsibility of the surgeon a sign‐out is performed in the OR, before the patient leaves the OR, in the presence of the full attending team. At least the following should be discussed and recorded: all essential information with respect to the operation performed; the results of the counting of used operation instruments and materials; and all items necessary for adequate postoperative care
8	Patient transport	During transport of the patient, bedrails are in the upright position or the patient is secured sufficiently
9	Handover to recovery room	The person responsible for patient transfer ensures the patient's vital parameters and leaves only once the patient has been adequately reconnected to monitoring equipment and shows stable vital signs
10	Discharge from recovery room	The postanaesthesia recovery score and pain score are documented in the patient's dossier at the time of discharge from the recovery room. Discharge of the patient from the recovery room is done once predetermined discharge criteria have been checked, recorded and met. The person responsible for discharge is documented in the patient's file
11	Discharge from the surgical ward	In the ward the postoperative pain score is measured and documented. The patient is discharged only once predetermined discharge criteria have been met and recorded. The person responsible for discharge is documented in the medical file. The patient is informed about the procedure performed and follow‐up treatment. Documentation on the surgical procedure is recorded in the medical file within 2 days after discharge from hospital
12	Safety of medical equipment	A date‐valid sticker concerning the support state of all medical equipment is visible
13	Counting used surgical equipment	Used surgical meshes, needles, instruments and disposables are counted by two people and the results are recorded in the patient's file
14	Behaviour with respect to infection prevention	The wearing of OR suits, clogs, surgical masks and caps, the handling of personal jewellery of attending team members, the performance of hand hygiene, OR door movements and preoperative removal of the patient's hair must be done according to existing national guidelines
15	Air ventilation and conditioning characteristics of the operating room	Adequacy of ventilation pressure in the OR must be displayed, with the facility to generate an alarm if malfunctioning. Positioning of the OR light and of the patient under the plenum must be done in accordance with optimal air‐conditioning characteristics for the prevention of wound infection
16	Prevention of transmission of hepatitis B and MRSA	Each institution has an assured procedure (in accordance with existing directives) for the prevention of transmission of hepatitis B and MRSA
17	Double‐check of medication	The preparation and administration of parenteral drugs are double‐checked
18	Propofol handling	Handling of propofol must be according to the directives of the Dutch Healthcare Inspectorate and manufacturer instructions
19	Drug storage	No date‐expired medication is stored in any local place that is in use for drug storage

OR, operating room; MRSA, methicillin‐resistant .

Table 2

Definition of the Joint Commission international patient safety goals16

IPSG	Goal	Standard
1	Identify patients correctly	The hospital develops and implements a process to improve the accuracy of patient identification
2	Improve effective communication	The hospital develops and implements a process to improve the effectiveness of verbal and/or telephone communication among caregivers
3	Improve the safety of high‐alert medications	The hospital develops and implements a process to improve the safety of high‐alert medications
4	Ensure correct site, correct procedure, correct patient surgery	The hospital develops and implements a process for ensuring correct site, correct procedure, correct patient surgery
5	Reduce the risk of healthcare‐associated infection	The hospital adopts and implements evidence‐based hand hygiene guidelines to reduce the risk of healthcare‐associated infection
6	Reduce the risk of patient harm resulting from falls	The hospital develops and implements a process to reduce the risk of patient harm resulting from falls

IPSG, international patient safety goal.

Several methods are available to measure patient safety and identify risks. Record review is the most frequently used and a widely accepted method of measuring incidence rates of adverse events17, 18. However, the reliability of record review is often considered only moderate, and there is no evidence that record review really detects all adverse events19. Direct observation methods of patient care capture valuable (real‐time), accurate and precise patient data20, 21. A recent study22 of unannounced JCI accreditation surveys showed that changes in practice during periods of surveyor observations in US hospitals decreased patient mortality significantly. Patient tracer observations are seen internationally as a robust method of measuring patient safety performance in daily practice23. In patient tracer methodology auditors follow representative patients from a specific hospital's patient population to evaluate compliance with standards16, 23. During each tracer procedure, the auditor(s) systematically observe and document care aspects according to: the course of care, treatment and services provided to in‐hospital patients; the interrelationships between and among disciplines and departments; and potential concerns in the patient care process.

The aim of the present multicentre observational pilot study was to develop a comprehensive and easy‐to‐use Surgical Patient safety Observation Tool (SPOT) to measure perioperative safety performance in daily clinical practice and to identify safety improvement areas. The development procedure included pilot testing of SPOT in eight hospitals to assess the clinimetric characteristics in line with previous studies24, 25, 26, 27. Feasibility was added to these characteristics (Table 3).

Table 3

Definition of clinimetric characteristics24, 25, 26, 27

Criterion	Definition	Score
Measurability	Measurable elements are measurable by observation	Good: at least 80 per cent of elements within the topics
		Moderate: more than 20 to less than 80 per cent of elements within the topics
		Poor: 20 per cent or less of items within the topics
Applicability	Measurable elements are applicable to the selected patients/procedures.	Good: at least 80 per cent of elements within the topics
		Poor: less than 80 per cent of elements within the topics
Improvement potential	Room for improvement of current practice (topic level)	Good: compliance with the standard less than 90 per cent
		Poor: compliance with the standard at least 90 per cent
Discriminatory capacity for comparison	Discrimination of practice performance (compliance with the standards) between different topics and between departments or hospitals	Good: more than 20 per cent variation between lowest and highest scores
		Poor: 20 per cent or less variation between lowest and highest scores
Feasibility	Easy to use and applicable without the help of others (after instruction)	Good: at least 90 per cent of the auditors
		Moderate: 50–90 per cent of the auditors
		Poor: less than 50 per cent of the auditors

Methods

No ethical approval was considered necessary, as no intervention was performed for a specific patient care process. A RAND‐modified Delphi procedure28, 29 was performed to reach consensus on the content of a standardized SPOT. The procedure consisted of the following steps.

Recruiting hospitals and collecting surgical patient safety observation checklists

The SPOT study was introduced in 2014 to all eight Dutch academic hospitals associated with the Netherlands Federation of University Medical Centres (NFU) and to all 14 member hospitals of the Dutch Safe Curative Care Association (Vereniging Veilige Curatieve Zorg; VVCZ), which includes one NFU academic hospital. VVCZ member hospitals help one another to improve patient safety. All 21 hospitals (8 academic, 7 tertiary care and 6 regional care hospitals) were invited and agreed to participate in the study. The hospitals were asked to share locally used surgical patient safety observation tools and checklists.

First Delphi round: item selection by experts

From the collected hospital checklists, items covering one or more of the 19 perioperative patient safety observation topics based on national and international perioperative guidelines, and used by the IGZ, were selected. These items, grouped per topic, were digitally presented to a multidisciplinary group of experts (professionals selected by the participating hospitals with specific interest and qualifications in perioperative patient safety, and locally identified as experts in this area). In total, 13 perioperative experts participated in a first Delphi round: five physicians, five operating room (OR) staff members, two ward nurses and one medical technician (Table S1, supporting information). These experts received a preselected list of topics depending on their specific expertise and relevance to their function. They ranked the observation items by making a personal top three list per topic. As well as ranking, the experts were also invited to comment on the content and phrasing of the items. The results of this step were used for reaching consensus on content and phrasing of the items for the final observation tool.

Second Delphi round: discussing the results of the first Delphi round

The scores and suggestions for phrasing of the items by the expert panel were processed and analysed. Observation items rated in the top three ranking received 10, 5 or 2 points respectively. The number of experts eligible to score topic items ranged from four to seven per topic. Thus, the potential maximum score of a topic item ranged from 40 to 70 points (Table S1, supporting information). All items that scored at least 50 per cent of the maximum were presented in a face‐to‐face consensus meeting. To create broad support, a second expert panel of ten experts in perioperative patient safety from VVCZ member hospitals (3 OR managers, 1 hygiene and infection adviser, 2 quality and safety managers, 1 anaesthesia nurse, 1 OR nurse, 1 physician and professor in perioperative patient safety, and 1 senior researcher – NFU member hospitals were not able to participate in this second Delphi round) discussed the results in order to achieve consensus regarding the content of the final SPOT tracer list.

Pilot testing of the Surgical Patient safety Observation Tool

The SPOT tracer list was pilot‐tested in one academic, four tertiary care and three regional care VVCZ member hospitals between September 2015 and April 2016 to assess the clinimetric characteristics of measurability, applicability, improvement potential, discriminatory capacity and feasibility according to predefined criteria (Table 3). Eight experienced auditors (3 physicians and 5 non‐physicians) assessed the feasibility and applicability of SPOT. For this last assessment, auditors responded to two statements – ‘SPOT is easy to use’ and ‘I am able to use SPOT independently’ – using a 6‐point answering scale (1, totally disagree; 6, totally agree). To facilitate interpretation, the answer categories of 5 (agree) and 6 (totally agree) were combined to express good feasibility.

Results of the observations are presented in a descriptive way, as numbers and percentages. The various scoring definitions of the clinimetric characteristics are presented in Table 3.

Results

Participating hospitals and local surgical patient safety observation checklists

Twenty‐one hospitals joined the study and delivered their local perioperative observation checklists covering a large variety of observation items related to perioperative patient safety. Four hospitals (2 academic, 1 tertiary care and 1 regional care hospital) did not use a perioperative patient safety observation checklist, but were performing specific in‐depth audits on different patient safety topics such as ‘handover’, ‘medication safety’ and ‘infection prevention’. The other 17 hospitals (6 academic, 6 tertiary care and 5 regional care hospitals) did use a perioperative patient safety observation checklist for internal auditing (Table S2, supporting information). The topics ‘behaviour with respect to infection prevention’ (15 hospitals), ‘time‐out before the operation’ (14) and ‘double‐check of medication’ (13) were most frequently represented in the obtained checklists. Least represented topics were ‘intraoperative team communication’ (3) and ‘discharge from the surgical ward’ (3) (Table S2, supporting information).

First Delphi round: item selection

In the first Delphi round, 13 experts independently and anonymously prioritized the observation items by means of a personal top three per topic. Items in two topics, ‘safety of medical equipment’ and ‘counting used surgical equipment’, failed to reach the 50 per cent criterion (the cut‐off point for discussion in the second Delphi round) because of large variation in experts' ratings. For these topics, considered too important to exclude, items having at least 20 per cent of the maximum score were taken into the second Delphi round. In total, 24 items (range 1–6 items per topic) were presented to the expert panel for discussion.

Second Delphi round: face‐to‐face meeting

The second Delphi round resulted in consensus on the content of observation items per topic. Based on the results, a tracer list of 134 measurable elements (MEs), with a range of 1–16 MEs per topic, was designed (Appendix S1, supporting information). For practical reasons related to use of the SPOT inside or outside the physical OR complex, SPOT was divided into two parts: part A for tracing the preoperative and peroperative phase (including holding area or preanaesthesia care unit (22 MEs), OR (65 MEs) and recovery room (39 MEs)), and part B for tracing the postoperative discharge process on the surgical ward (8 MEs). Some MEs were measured twice in different locations of the perioperative patient path, and therefore given a unique number to be reviewed and scored. This was the case for the topics ‘patient transport’ (measured before and after the operation), ‘behaviour with respect to infection prevention’, ‘safety of medical equipment’ and ‘double‐check of medication’ (all measured in both the operating and recovery room). In addition, a dichotomous scoring scale, ‘yes’ or ‘no’ (respectively complied or not complied to the standard) per measurable item, was added to the list, facilitating an overall score calculation per topic. Finally, an instruction guide was developed and added to the SPOT.

Results of the pilot test

A total of eight hospitals participated in the SPOT pilot test, resulting in 2085 observations and scores of measurable elements related to 36 surgical procedures over ten surgical specialties: general surgery, cardiothoracic surgery, obstetrics and gynaecology, neurosurgery, ophthalmic surgery, plastic surgery, urology, oral and maxillofacial surgery, orthopaedic surgery, and ear, nose and throat surgery. All 19 patient safety topics were found to be measurable (adequate information available by observation), ranging from eight (‘discharge from the surgical ward’) to 291 (‘time‐out before the operation’) MEs among topics (Table 4).

Table 4

Overall results of the pilot Surgical Patient safety Observation Tool scores per topic

			Compliant†
Perioperative patient safety topic	Observations*	Applicable	Yes	No
Communication and handover
1 Patient identification	127	127 (100)	118 (91·5)	11 (8·5)
2 Handover from wards	95	92 (97)	73 (74)	25 (26)
3 Pre‐time‐out before the operation	204	24 (11·8)	23 (96)	1 (4)
4 Time‐out before the operation	291	288 (99·0)	275 (92·9)	21 (7·1)
5 Intraoperative team communication	49	41 (84)	41 (100)	0 (0)
6 Handover at shift change during the operation	48	18 (38)	18 (100)	0 (0)
7 Sign‐out after the operation	142	127 (89·4)	120 (93·8)	8 (6·2)
8 Patient transport	33	33 (100)	28 (85)	5 (15)
9 Handover to the recovery room	190	180 (94·7)	161 (88·5)	21 (11·5)
10 Discharge from recovery room	179	172 (96·1)	157 (91·3)	15 (8·7)
11 Discharge from the surgical ward	8	8 (100)	8 (89)	1 (11)
Medical equipment
12 Safety of medical equipment	33	33 (100)	30 (86)	5 (14)
13 Counting used surgical equipment	114	108 (94·7)	95 (87·2)	14 (12·8)
Infection prevention
14 Behaviour with respect to infection prevention	262	258 (98·5)	227 (81·4)	52 (18·6)
15 Air ventilation and conditioning characteristics of the OR	18	18 (100)	15 (83)	3 (17)
16 Prevention of transmission of hepatitis B and MRSA	36	32 (89)	26 (81)	6 (19)
Medication
17 Double‐check of medication	116	116 (100)	100 (72·5)	38 (27·5)
18 Propofol handling	49	29 (59)	28 (97)	1 (3)
19 Drug storage	91	88 (97)	83 (94)	5 (6)
Total (mean)	2085	1792 (86·9)	1626 (88·6)	232 (11·4)
Median (range)	95 (8–291)	97 (11·8–100)	89 (72·5–100)	11 (0–27·5)

Values in parentheses are percentages unless indicated otherwise.

Observed measurable elements.

Yes and No numbers combined are sometimes higher than the number of observations per topic, because both answer options were sometimes scored by the auditors. OR, operating room; MRSA, methicillin‐resistant .

The topics showed good applicability (median 97 (range 11·8–100) per cent). Poorly applicable topics were ‘pre‐time‐out before the operation’ (applicable in only 12 per cent of procedures), ‘handover at shift change during the operation’ (applicable in 38 per cent of procedures) and ‘propofol handling’ (a strict prescription of how to handle the intravenous administration of propofol to prevent bacterial contamination30, used in only 59 per cent of procedures) (Table 4).

The overall improvement potential of the topics was good: median 89 (range 72·5–100) per cent compliance per topic. Ten patient safety topics scored less than 90 per cent compliance, including ‘handover from wards’ (compliance 74 per cent) and ‘double‐check of medication’ (compliance 72·5 per cent) (Table 4).

The perioperative patient safety compliance score per topic (range 72·5–100 per cent) also indicated good discriminatory capacity of the tool (variation 27·5 per cent), with greatest variation for topics 2 and 17 (‘handover from wards’ and ‘double‐check of medication’) (Table 4).

The overall patient safety performance with respect to the participating hospitals showed lower variation (range 13 per cent), with greatest variation between hospitals 3 and 5, which scored 83 and 96 per cent respectively (Table 5).

Table 5

Total ‘yes’ scores for compliance per hospital per topic

	Participating hospitals
Surgical patient safety topic	1	2	3	4	5	6	7	8	Range (%)	Total ‘yes’
Communication and handover
1 Patient identification	70 (92)	12 (93)	8 (86)	7 (100)	3 (100)	6 (92)	5 (100)	7 (93)	86–100	118 (91·5)
2 Handover from wards	39 (77)	9 (90)	2 (20)	5 (100)	4 (100)	5 (60)	5 (100)	4 (80)	20–100	73 (74)
3 Pre‐time‐out before the operation	–	–	–	9 (90)	5 (100)	–	9 (100)	–	90–100	23 (96)
4 Time‐out before the operation	181 (96·3)	29 (94)	26 (82)	16 (81)	12 (100)	11 (100)	–	–	81–100	275 (92·9)
5 Intraoperative team communication	28 (100)	–	5 (100)	3 (100)	2 (100)	3 (100)	–	–	100	41 (100)
6 Handover at shift change during the operation	9 (100)	–	3 (100)	3 (100)	3 (100)	–	–	–	100	18 (100)
7 Sign‐out after the operation	90 (93)	7 (100)	7 (100)	7 (100)	5 (100)	4 (88)	–	–	88–100	120 (93·8)
8 Patient transport	16 (80)	2 (100)	3 (100)	2 (100)	2 (100)	2 (100)	–	1 (100)	80–100	28 (85)
9 Handover to the recovery room	99 (83)	12 (100)	10 (100)	12 (100)	9 (100)	11 (100)	–	8 (88)	83–100	161 (88·5)
10 Discharge from recovery room	114 (89·3)	15 (100)	–	14 (93)	–	–	–	14 (100)	89·3–100	157 (91·3)
11 Discharge from the surgical ward	–	–	–	–	–	–	–	8 (89)	89	8 (89)
Medical equipment
12 Safety of medical equipment	17 (81)	3 (100)	2 (67)	2 (100)	2 (100)	2 (100)	1 (100)	1 (100)	67–100	30 (86)
13 Counting used surgical equipment	62 (86)	–	13 (93)	7 (100)	6 (86)	7 (93)	–	–	86–100	95 (87·2)
Infection prevention
14 Behaviour with respect to infection prevention	145 (84·1)	15 (68)	21 (81)	14 (93)	13 (85)	14 (86)	3 (83)	2 (67)	67–93	227 (81·4)
15 Air ventilation and conditioning characteristics of the OR	9 (75)	1 (100)	2 (100)	1 (100)	1 (100)	1 (100)	–	–	75–100	15 (83)
16 Prevention of transmission of hepatitis B and MRSA	17 (92)	2 (50)	2 (50)	2 (100)	2 (100)	1 (100)	–	–	50–100	26 (81)
Medication
17 Double‐check of medication	68 (78)	8 (100)	3 (29)	8 (88)	4 (100)	6 (83)	2 (75)	1 (25)	25–100	100 (72·5)
18 Propofol handling	18 (96)	3 (100)	2 (100)	3 (100)	–	2 (100)	–	–	96–100	28 (97)
19 Drug storage	57 (97)	7 (83)	5 (100)	6 (100)	1 (100)	2 (100)	2 (67)	3 (100)	67–100	83 (94)
Total (mean)	1039 (88·6)	125 (91·4)	114 (82·5)	121 (94·3)	74 (96)	77 (92)	27 (90)	49 (87)		1626

Values in parentheses are percentages unless indicated otherwise. OR, operating room; MRSA, methicillin‐resistant .

All eight auditors considered SPOT easy to use, with scores of 5 or 6 points (maximum 6). Seven of the eight auditors stated that they were able to use SPOT independently. One auditor suggested the addition of a ‘partly complied’ option to the dichotomous scoring system.

Discussion

A comprehensive easy to use surgical patient safety observation tool, SPOT, was developed and pilot‐tested for prospective risk analysis, monitoring, benchmarking and improving perioperative safety. SPOT is currently used in all VVCZ member hospitals as part of their intrahospital and yearly interhospital auditing. The audit team consists of an audit leader together with two expert professionals in perioperative care (one physician and one non‐physician). The hospitals use SPOT results to support internal perioperative patient safety improvement initiatives. Results are anonymized and then used for benchmarking between VVCZ member hospitals.

A variety of local in‐hospital observation tools were found, and so a RAND‐modified Delphi technique was used to reach consensus on the content of a comprehensive and easy‐to‐use tracer and observation tool28, 29. The present study resulted in a measurement instrument for perioperative patient safety performance in daily clinical practice. All 19 national perioperative patient safety topics appeared to be easily measurable with SPOT, and seem generally applicable to various surgical procedures.

The overall improvement potential appeared to be good; in particular, topics such as ‘handover from wards’ and ‘double‐check of medication’ showed much room for improvement. At topic level, good discriminatory capacity of the tool was shown. Overall scores showed less variation between the hospitals that participated in the pilot.

The chosen cut‐off point of 90 per cent indicating good or poor compliance, as used in previous studies24, 25, 26, 27, may need revision in the future. It could be argued that 100 per cent compliance should be the goal for all patient safety topics.

In the absence of direct observation of daily practice, true compliance of standard perioperative procedures is unknown31. Auditing is thus an important activity of quality management, used to explore whether daily care is consistent with evidence‐based guidelines. Observation in daily practice by healthcare professionals themselves with the use of SPOT is considered a simple and therefore attractive method to stimulate patient safety guideline compliance. Insight in actual provided patient safety is a crucial step in the plan–do–check–act cycle before selecting and realizing patient safety improvement activities32. Furthermore, detailed monitoring and feedback about current non‐optimized patient safety performance contribute to employee awareness of the risky nature of their actions and motivate them in changing behaviour33.

Worldwide, patient safety programmes are designed to measure and improve safety in order to control risks and minimize potentially avoidable patient harm, by systems and teamwork approaches34. Supplementary to the legal frameworks and professional guidelines, the Dutch basic requirements for a hospital Safety Management System (SMS) are nationally established and described in the Dutch Technical Agreement (DTA)35. The DTA contributes to national uniformity of the SMS and aims to create transparency in patient safety in hospitals. Dutch hospitals have to meet the need for transparency by having their SMS evaluated and audited by internal and external parties35. In that way, the public and policy‐makers emphasize taking action to ensure compliance with safety guidelines. However, external pressure may also be associated with negative consequences, and may be more focused towards better administration than actually improving patient safety. It is therefore important to assess what really matters and focus on those patient safety issues that actually improve patient safety and motivate frontline caregivers36.

No perioperative patient safety observation tool similar to SPOT is currently available to characterize the safety of daily clinical practice. Perioperative experts from various hospitals participated in the present study, which resulted in broad support for one comprehensive and condensed observation tool. It offers a structured method to measure and monitor perioperative risks to improve patient safety. From the pilot test it was shown that SPOT was a suitable easy‐to‐use tool to identify patient safety risks in the perioperative process. The SPOT concept can be transferred easily to a broad range of medical disciplines and activities with an interventional character.

The instrument is currently set up with ‘yes’ and ‘no’ as possible answers for included items. This turned out to be insufficient for some observations in clinical practice. One auditor suggested that ‘partly complied’ be added as a third answer category; this should be taken into account in further development. A limitation of SPOT might be the subjective judgement of a single observation if SPOT is used by only one auditor. Preferably, the observations are done by at least two auditors, although this might interfere with a maximum allowable number of professionals being in the OR. For this reason, during an audit, the results of an individual auditor should be discussed with the audit team. In addition, the presence of an observer may influence the normal daily practice behaviour, which may result in an overestimation of the actual performance (the Hawthorn effect), although the size of this effect is not known37. Another concern, despite the simplicity of SPOT, is the potential time‐consuming nature of processing and analysing the data, which may be a barrier to use. To facilitate this, SPOT should be transformed from a paper version into a digital ‘SPOT tracer and monitor application’, as has already been realized in Radboud University Medical Centre. Although SPOT is based on international goals, national guidelines and the perioperative patient safety framework of the Dutch Health Care Inspectorate15, the IGZ framework did not cover the latest developments, on such topics as ‘hand hygiene’ and ‘air ventilation and conditioning characters of the operating room’, so new items should be addressed, while other items will need reframing.

Best practice performances of departments and hospitals should enable them to share experiences and learn from one another. The anticipated users of SPOT are internal auditors, such as front‐line professionals, and external auditors, such as peer colleagues from other hospitals. Based on these initial experiences with SPOT in the Netherlands, a regular update of the content of the tool is recommended. This may be done in cooperation with national guideline developers and with use of international standards, such as the JCI accreditation standards. Because SPOT is based on international patient safety guidelines, the tool could be widely applicable. Evaluation of SPOT outside the Netherlands should therefore be undertaken to assess whether this is the case. Further work should include evaluation of the effect of using this condensed and standardized tool on patient safety outcomes.

Table S1 Number of perioperative experts who judged perioperative patient safety topics

Table S2 Patient safety observation topics covered in local observation tools of the participating hospitals

Appendix S1 Surgical Patient safety Observation Tool (SPOT)

Click here for additional data file.