An easy, prompt and reproducible methodology to manage an unexpected increase of incident reports in surgery theatres.

OBJECTIVES: Surgery is a high-risk hospital area for adverse events (AEs) occurrence. This study aims to develop an effectiveness and reactive methodology to manage an unexpected increase of AEs in the operating rooms (ORs) of a large Academic Hospital providing about 30 000 surgeries per year.
METHODS: The study included three phases: 1. analysis of the AEs collected through the hospital incident reporting system from 2014 to 2015; 2. development of a programme to improve the surgical patient's safety and 3. application and evaluation of the programme effectiveness.
RESULTS: In 2014, all hospital AEs were 825 (10.3% in ORs), while in the first 5 months of 2015, they were 645 (17.7% in ORs) [relative risk (RR) 2015 vs 2014=1.7; 95% CI=1.3 to 2.2; p<0.0001] with two sentinel events. Due to this increase, 177 real-time observations were planned in 12 ORs with external staff (n.25) during 1 week in June, July and November 2015 using a checklist with 14 items related to the patient's pathway (surgical site, time-out, medical records and sponges count). After the observations, the AEs decreased from 11.4×1000 surgeries (January-June 2015) to 8.6×1000 (July-December 2015) (RR=0.7, 95% CI=0.6 to 0.9, p<0.05). Compliance to the correct procedures applied by ORs staff has improved during the year for all items.
CONCLUSIONS: The methodology of this study has been revealed effective to control an unexpected increase in AEs and to improve the healthcare workers' adherence to correct procedures and it could be translated in other patients' safety settings.

Introduction

An adverse event (AE) is defined as an injury or harm to a patient that is caused by health professional management, rather than an underlying disease. Surgery is one of the high-risk areas for the occurrence of AEs. 1–4Literature reports AEs rates between 7% and 37% in general surgery. 1 5 These complications include wrong patient/procedure/site surgery, anaesthesia equipment problems, lack of availability of necessary instruments, unanticipated blood loss, non-sterile materials and surgical items (eg, sponges) left inside patients. 6

Many different studies highlight the problem that no single tool of risk management (RM), such as safety checklists and incident reporting system (IRS), can be effective as a standalone instrument, but unfolds its effect only when embedded in a subordinate RM system which integrates tailor-made elements to increase patient safety into the workflows of each organisation.7

We summarise below what is currently known about the two tools (surgical checklist and IRS) described in relevant previous studies.

Since 2008, in all countries, the most used instrument to prevent errors and complications, which may occur during surgery or perioperatively is the surgical checklist.8 Studies have suggested that checklists may reduce errors for many reasons, such as ensuring that all critical tasks are carried out, encouraging a non-hierarchical team-based approach, enhancing communication, catching near-misses (NMs) early, anticipating potential complications and having technologies to manage anticipated and unanticipated complications.6 Recently, however, a review of nearly 7000 surgical procedures performed in five English hospitals found that the checklist was used in 97% of cases, but they were completed only 62% of the time.9When the researchers observed a smaller number of procedures, they found that practitioners often failed to give the checks their full attention and read only two-thirds of the items out loud.10

With regard to the IRS, despite its widespread implementation, it is not clear whether incident reporting has resulted in improvements to safety and its utility has recently been extensively debated. Critiques of IRS suggest that its role in managing safety has been overemphasised and call for less emphasis on counting incidents and more emphasis on the effective analysis of incidents and organisational learning.11–18 Taking the example of wrong-site surgeries, the Veterans Affairs have developed a tool to reduce their risk, and in 2004, Joint Commission (JC) required that hospitals implement a time-out to prevent these events. Despite implementation of these interventions, the apparent rate of wrong-site surgeries from IRS continues to grow. This increase is much more likely due to increased reporting (reporting bias) from increased awareness rather than an increase in wrong-site surgery from all these interventions. These data urge caution in interpreting changes over time in reported events, even highly visible events that are well defined.19

Furthermore, the standardisation of surgical processes should not be limited to the operating room (OR): several studies have shown that the majority of surgical errors (53%–70%) occur outside the ORs, before or after surgery, making it likely that a more substantial improvement in safety could be achieved by targeting the entire surgical pathway.20–22 For example, de Vries et al developed the Surgical Patient Safety System checklist, a multidisciplinary checklist that follows the surgical pathway from admission to discharge.

The purpose of our project was to develop a new, easy, prompt and reproducible methodology to manage and reduce an unexpected increase of incident reports linked to surgery processes. In this report, we described the strategy adopted in a large Academic Hospital (AH) and consisted of combining several tools from IRS as a trigger to alert on the basis of a statistically significant increase of AEs and sentinel events to real-time observations using a specific checklists built defining first the surgical pathway.

Methods

Contextual elements considered important at the outset of introducing the project were the type of involved hospital, the structural and organisational changes introduced recently, the IRS active in the AH for several years and the procedure in use to assure the surgical patient’s safety.

The AH in which the study was conducted is an Italian large urban high specialisation hospital where about 30 000 surgical procedures take place annually. Starting from December 2014, a large part of the surgical units moved to a new building and new operating theatres. This change also included new organisation since previously each surgical unit had its own ORs and wards, while in the new building, seven units were concentrated in two blocks each with six ORs and common wards structured by intensity of care.

For several years in the AH an IRS is active. It consists of a volunteer and anonymous reporting of NM/AEs according to a modality of structured collection. It is still a paper-based system. Every incident that leads (or could have led) to a risk for patients, independently of the aspect if the incident could have been avoided or not and if it was based on misbehaviour of the healthcare staff or not, should be reported to the hospital RM team.23 This team daily analyses data from IRS, studies in depth the events, identifies the improvement actions leading to the reduction of future risk of similar events and on the basis of data from IRS, on-site checks and audit, root cause analysis routinely produces reports to units and top management.

During the years, written procedures about the surgical RM (patient identification, surgery checklist and sponge–sharps–instruments count) and the correct behaviours that should be respected by the OR healthcare workers (HCWs) were disseminated and implemented throughout the hospital. The surgical safety checklist used in the hospital, provided by the National Health Ministry, similar to that of the WHO checklist, is divided into three parts: before anaesthesia (‘sign in’), before skin incision (‘time-out’) and immediately after the end of the surgery (‘sign out’).24 25

The project was developed from June 2015 to December 2015 and it consisted of three phases:

The analysis of the incident reports about surgical procedures collected by the hospital IRS in the years 2014 and 2015 and the identification of an unexpected excess of incident reports and sentinel events occurred in ORs.

The development of a programme to reduce surgical safety risk events.

The evaluation of the effectiveness of the programme.

The project manager was the hospital risk manager assisted by the RM team (one physician, five nurses) and the site teams assembled and trained to conduct the real-time observations in the hospital operating theatres included two persons: one expert in on-site surveys and one medical resident.

For assessing the impact of the interventions, we chose three approaches: the analysis of the trends of the incident reports that occurred in ORs, the compliance to the single items of the surgical checklist used during the real-time observations in ORs and the recognition of the number of deaths after 30 days from a surgical procedure and of unplanned returns to theatre for all non-emergency surgical patients before and after the entire project.

Data collected from the real-time observations were entered in an Excel spreadsheet and were analysed using the statistical software IBMSPSS, V.20. P<0.05 was considered statistically significant. Other data, such as number of deaths after 30 days from a surgical procedure and of unplanned returns to theatre for all non-emergency surgical patients, were extracted from the AH databases used routinely.

We also added a list of critical circumstances reported by the personnel involved in the real-time observations in a narrative way.

Results

Phase 1: analysis of hospital IRS database and identification of outbreaks in surgical area

Table 1 shows the reported incidents occurred in the ORs in 2014 (January–December) and in 2015 (January–May), according to the classification of the JC International (JCI) standards.26

Table 1

Distribution of hospital-reported adverse events (AEs)/near-misses (NMs) occurred in the operating rooms (ORs) according to Joint Commission International standards in 2014 (Jan–Dec) and in 2015 (Jan–May)

Hospital incident reporting system	2014 Jan–Dec	2015 Jan–May	RR	95% CI	P value
	N°(%)	N° (%)
No. of AEs/NMs occurred in all hospital units	625	645
No. AEs/NMs occurred in the OR	85 (100.0)	23 (100.0)	1.7	1.3 to 2.2	<0.0001
1. Anaesthesia and surgical care	43 (50.6)	23 (20.4)	0.4	0.3 to 0.6	<0.0001
2. Ensure correct site, correct procedure, correct patient surgery	11 (12.9)	7 (6.2)	0.5	0.2 to 1.2	n.s.
3. Staff qualification and education	6 (7.1)	28 (24.8)	3.5	1.5 to 8.1	<0.01
4. improve the effectiveness of the communication among health workers	0 (0.0)	21 (18.6)	–	–	–
5. Governance, leadership and direction	2 (2.4)	18 (15.9)	6.8	1.6 to 28.4	<0.01
6. Prevention and control of infection	15 (17.6)	8 (7.1)	0.4	0.2 to 0.9	<0.05
7. Patient and family rights	8 (9.4)	6 (5.3)	0.6	0.2 to 1.6	n.s.
8. Sentinel events	0 (0.0)	2 (1.8)	–	–	–

RR, relative risk; CI, Confidence Intervals; n.s., not significant.

During the first five months of 2014, the percentage of incidents that occurred in the ORs on the total of hospital-reported incidents was 11.0% (40/365); in the same period (January–May) of 2015, it was 17.7% (113/645) [relative risk (RR) 2015 vs 2014=1.6; 95% CI=1.1 to 2.2; p<0.01]. Considering also the first five months, all incident reports across the organisation were 23×1000 admitted patients in 2014 and 43×1000 admitted patients in 2015.

Phase 2: development of a programme to reduce surgical safety risk events

As soon as the exceeding reported incidents were detected, the RM team set up an intervention programme to reduce the emerging surgical safety risk events based on:

Selecting the critical points during the surgical pathway from admission to discharge to ORs and to include them in a checklist to fill in during the real-time observations in the hospital operating theatres (see online supplementary appendix 1).

Development of checklists (available on request) according to existing guidelines and recommendations used in the hospital and in agreement with the international recommended standards26; the checklists used also had an open space to report significant episodes in a narrative way in order to facilitate the immediate understanding by all HCWs.

Assembly and training the site teams to conduct the real-time observations in the hospital operating theatres; each team included two persons: one expert in on-site surveys and one medical resident.

Planning of the real-time observations of a surgical intervention from the patient’s admittance in the OR to his/her exit.

The approach was structured in the following way:

Alert top management and set up of the measures to control the phenomenon.

Plenary session of surgery, anaesthesia and nurse chiefs where the state of the art, the NMs/AEs/sentinel events incidence, as well as the control programme were presented and discussed.

Feedback immediately after each real-time observation to the units in the way reported in figure 1.

Figure 1

The planning of the real-time observations conducted in the operating rooms withthe consequent corrective actions.

Phase 3: evaluation of the programme effectiveness

The real-time observations were conducted during a week in June, July and November 2015, including 177 surgical interventions at 12 surgical units (general, maxillofacial, ophthalmology, orthopaedic, plastic, urology, cardiac, intervention radiology, neurological, spinal, thoracic and vascular) with 25 surveyors.

The trends, stratified by quarter during 2015, that occurred in ORs and reported by all hospital HCWs and the percentages of those reported by only the OR HCWs (figure 2A) and the same trends according to the number of surgeries executed in the hospital (figure 2B); after the observations, the reported incidents in ORs decreased from 19.0% (143/753) to 16.9% (102/605) (RR=0.9; 95% CI: 0.7 to 1.1; P=0.3) and from 11.4×1000 surgeries (January–June 2015) to 8.6×1000 (July–December 2015) (RR=0.7, 95% CI=0.6 to 0.9, p<0.05).

Figure 2

Trends of incidents (reported by all healthcare workers and by only surgical teams personnel) stratified by quarter during 2015 ((A) percentage; (B) x 1000 surgeries).

The compliance to the single items of the checklist used during the real-time observations in ORs in June, July and November 2015 (table 2).

Table 2

Compliance to the single items of the checklist used during the real-time observations in operating rooms (ORs) in June, July and November 2015

Items	First survey week (June)	Second survey week (July)	Third survey week (Nov)	P value	P value
	% (N°/Tot.)	% (N°/Tot.)	% (N°/Tot.)	Jun–Nov	Jul–Nov
Preoperative items on ward
Surgical site is marked with ‘YES’	64 (38/59)	79 (23/29)	84 (16/19)	n.s.	n.s.
Preoperative items are verified	96 (74/77)	100 (45/45)	100 (32/32)	–	–
Patient’s surgical consent is correctly filled	*	96 (49/51)	100 (36/36)	–	–
Patient’s anaesthetic consent is correctly filled	*	94 (30/32)	100 (35/35)	–	–
Preoperative in OR
Time-out has been performed	68 (53/78)	90 (45/50)	89 (31/35)	<0.05	n.s.
Surgical checklist is present in medical records	98 (82/84)	100 (48/48)	89 (39/39)	–	–
Perioperatory folder is present in medical records	*	100 (46/46)	100 (39/39)	–	–
Uniform and headdress are clean	*	65 (33/51)	84 (31/37)	–	<0.05
Surgical masks are correctly worn	*	73 (37/51)	78 (29/37)	–	n.s.
Preoperative in recovery room
Gauzes count is correct	86 (48/56)	96 (25/26)	100 (27/27)	–	–
At discharge from the OR
Surgical checklist is correctly filled	90 (55/61)	90 (26/29)	97 (29/30)	n.s.	n.s.
Perioperatory folder is correctly filled	*	80 (28/35)	90 (26/29)	–	n.s.
Total	87†	51†	39†

*Critical items added to the checklist on the basis of the first site visit.

†In case of not applicability, it was not possible to evaluate all items in all the interventions.

The number of deaths after 30 days from a surgical procedure and of unplanned returns to theatre for all non-emergency surgical patients before and after the entire programme (table 3).

The programme effectiveness was evaluated at three levels

Narrative of the visits in ORs

The real-time observations allowed the opportunity to watch operating theatre staff during their routine surgical activity and to report critical circumstances such as the following:

Perioperative record was filled only after the end of surgery.

The medical part of the perioperative record was not filled. The surgeon justified that it was an emergency but the patient was hospitalized for two days.

An informed consensus was signed by the patient but it was not filled out.

The anaesthesiologist performed the identification of an elderly patient by suggesting him the name, the surname and the date of birth: the patient nodded.

Time-out checklist was used as a ‘check-box’ exercise without thinking about the patient safety.

Someone of the ORs staff managed time-out with disdain or theatricality.

During the inspections, some OR teams demonstrated that they had never done time-out before.

During the time-out team introduction when the checklist leadership asked ‘Do we know each other?’ the surgeon replied: ‘Do you want my the identity card?

Discussion

This study provides a new easy, prompt and reproducible strategy to manage an unexpected increase of surgical incident reports in a hospital and to assure the patient safety in the ORs.

The real-time observations covering the entire surgical pathway in ORs was the key point of the programme. The teams were fundamental to identify areas of improvement in implementation and enable provision of comprehensive feedback to ORs teams.

The first effect of the programme was a significant decrease of incident reports per 1000 surgeries when comparing the two periods before and after the observations in the ORs. It is hard to know whether this decrease in reported incidents was an authentic signal of lower error or simply a function of reporting bias or random variation through time.27.

At the same time, the incidents reported by the OR HCWs in the surgical theatre increased. This phenomenon suggests that the programme increased the sensitivity of the surgical personnel to reporting AEs/NMs, although a new IRS (such as electronic) has not been introduced.

This growth in sensitivity about the patient safety could be ascribed to the real-time observations in the ORs simultaneously with education provided by external personnel and the immediate (within 24 hours) email with the visit findings sent to the medical and nursing chiefs of the surgical teams. Also the plenary session, in which the hospital director presented the progressive steps (figure 2) in the management of the visits findings helped to improve the compliance to the correct surgical procedures. This was confirmed by the fact that none of the surgical teams received second or third alerts.

The distribution of the incident report topics in the considered periods (2014 and 2015) showed a shift of the critical issues in the fields of communication, staff qualification and education. The problem emerged because, during the last months of 2014 and the first months of 2015, surgical units were transferred from the old buildings to the new ones. The operating blocks and nursing teams were in common among the various surgical specialties contrary to before when each one had its own OR and own nursing staff. Therefore, the new activity organisation, the lack of personnel training and working with unknown colleagues in a different setting created opposition, conflict and difficulty in communication among HCWs. These problems generated an increase of anonymous reported incidents regarding the lack of adherence to hospital procedures in ORs presumably not by the person who committed the error, but by his/her colleague who detected it. At the same time, the incident reports in the field of ‘governance, leadership and direction’ increased. This fact was a failure signal of the hospital leadership in staff preparation for the transfer in the new building. Therefore, hospital personnel perceived a lack of organisation and coaching in dealing with a significant working change requiring flexibility, adaptability and participation.

The effectiveness of the research programme was demonstrated by the comparison among the compliance to the single items of the checklist used during the real-time observations in the ORs in three different periods: the first visits (June 2015), the second visits (July 2015) and the third visits (November 2015). Compliance to the correct procedures applied in the OR regarding the surgical RM and the correct behaviours to be respected by the HCWs has improved over time. A total adherence (100% of the real-time observations) was reached for patient’s surgical and anaesthetic consents correctly filled out, for the presence of surgical checklists and perioperative folder in medical records and for the correct counting of gauzes.

Surgical process deviations, that did not obtain the total adherence in the last visits, concerned the incorrect or absent marking of the surgical site, the performing of time-out and the surgical checklist and perioperative folder incorrectly filled out at patient discharge from the OR. The problem of adherence to surgical checklists is known in literature.28 In this research, the more frequent barriers to effective implementation included confusion regarding practical aspects of checklist use and the beliefs and attitudes of participating staff, particularly surgeons.

Also, the items of clean uniform headdress and surgical masks correctly worn remained critical behaviours even if, during the 2015 year, the HCWs demonstrated a higher attention to use.

The evaluation of the programme from hospital informative database did not demonstrate an increase or decrease in terms of complications or deaths after surgeries before and after the real-time observations. These results highlight the importance of the IRS as a valid trigger tool because only in the long term it is possible to detect statistically significant changes in clinical outcomes such as mortality or complications.

This study provides some important tips in the RM of the surgical patient. First, the IRS proved to be a trigger to alert the RM team on the basis of a statistically significant increase of reporting incidents and sentinel events occurring in surgical theatres. Also, JCI and Italian National Ministry recommend that the appropriate response by a hospital, even for a single sentinel event, includes conducting a timely, thorough and credible analysis developing an action plan designed to implement improvements to reduce risk.26 29 Second, the checklists for the real-time observations were demonstrated as an useful reactive and iso-resources tool to observe and to measure the routine process of all surgical specialty teams. Furthermore, the suspicions merged with the IRS were confirmed with these on-site surveys through the observed weaknesses and the reported narrative. Third, the described surgical pathway and its items were demonstrated as immediately comprehensive with respect to the JCI classification. The selected items focused on tasks that involved the whole team and all tasks were ultimately the responsibility of the team. The underlying concept is that ‘when a team fails to detect or correct an individual’s error it becomes a team error’.30 Therefore, a fundamental intervention to increase surgical patient safety should be to improve the quality of teamwork in the OR.31

This research presents the following limitation. As the case with observational study of this sort, there has been the possibility of a Hawthorne effect skewing results: team members behaving unnaturally, often improving their performance, due to the presence of an observer.26

In conclusion, the study provides a new effective and reactive methodology to control an unexpected increase in AEs and to improve the HCWs’ adherence to correct procedures. The programme could be translated into other patients’ safety settings of both large and small hospitals due to its simplicity and without adding resources.

Table 3

Percentages of deaths after 30 days from a surgical procedure and of unplanned returns to theatre for all non-emergency surgical patients stratified by quarter during 2015

Period	30 days mortality after surgeries
	N°, of operated patients	%	RR	95% CI	p value
First quarter 2015	2506	1.64	–	–	–
Second quarter 2015	2469	2.19	1.34	0.88 to 2.04	n.s.
Third quarter 2015	2322	2.02	1.24	0.80 to 1.91	n.s.
Fourth quarter 2015	2457	2.20	1.24	0.80 to 1.91	n.s.

Period	Returns to theatre for all non-emergency surgical patients
First quarter 2015	2506	0.88	1.55	0.76 to 3.18	n.s.
Second quarter 2015	2469	0.57	–	–	–
Third quarter 2015	2322	0.90	1.59	0.78 to 3.30	n.s.
Fourth quarter 2015	2457	0.69	1.22	0.57 to 2.61	n.s.

RR, relative risk; n.s., not significant.