WHO Safer Surgery checklist compliance amongst paediatric emergency plastic surgery patients in an UK hospital.

INTRODUCTION: The WHO Safer Surgery checklist has become an important component of perioperative safety. Our objective, was to determine the compliance of completing the checklist for paediatric emergency plastic surgery patients at our unit.
METHODS: An initial baseline was performed with 70 patients over two months at our unit. Following this, we raised awareness at an audit meeting and closed the audit loop using 80 patients over two months. The audit is reported in line with SQUIRE 2.0 criteria.
RESULTS: Initial compliance was 88% overall and this increased to 91% post-intervention. Compliance with the individual stages in both cycles was for sign-in: 85%-86%, for time-out 92%-98% and for sign-out 86%-89%. Around one in four checklists were not scanned in both periods.
CONCLUSION: This audit showed a high overall level of compliance in the checklists that were scanned and available for scrutiny. We have identified the areas that most need improvement and suggest ways for doing so.

Introduction

Surgical morbidity and mortality are concerns for public health. In 2012, it was estimated that between 266 and 359 million major surgical operations were performed worldwide [1]. To break this down further, of the estimated 234 million major surgical operations in 2004 [2], it is estimated that major morbidity complicated 3–16% of cases. In this study, the rate of permanent disability or death was between 0.4 and 0.8% and it is reported that almost half of the adverse events were deemed to be preventable [3], [4].

In June 2008, the World Health Organization (WHO) developed a Safer Surgery checklist to be used globally to improve and ensure perioperative safety for patients. This checklist was designed with the intention of improving teamwork between operating room staff and ensuring the consistent use of a perioperative safety process. It includes various items around three perioperative stages: sign-in (before induction of anaesthesia), time-out (before skin incision) and sign-out (before the patient leaves the operating room). In each stage members of the team have to confirm that certain tasks have been done before the team can proceed. The pivotal multicenter before and after study was performed by Haynes et al. involving 3733 consecutive patients before and 3955 patients after implementation of the checklist. They showed a near 47% drop in mortality (from 1.5% to 0.8%) and a 36% decrease in morbidity [5] and this was validated by further work [6].

The checklist was promptly implemented in multiple health care systems around the world including the NHS in 2009 [7]. However, some studies have suggested that compliance in completing the checklist is minimal [8], [9]. Monitoring the implementation and compliance with safety and quality assurance systems is an important aspect of clinical audit and surgical surveillance [10]. Data on WHO checklist compliance for a subset of our patients was lacking locally.

Our objective, was to determine the compliance of completing the WHO Safer Surgery checklist for paediatric emergency plastic surgery patients at our unit. Our study is reported in line with SQUIRE 2.0 guidelines [11].

Methods

This quality improvement study was performed at the Evelina London Children's Hospital (ELCH). The ELCH is one of only two specialist children's hospitals in London, along with Great Ormond Street Hospital. The neonatal unit cares for 900 babies a year and overall the hospital cared for 55,000 children in 2016 [12]. We assessed emergency paediatric plastic surgery patients operated on during the period July–August 2016 to establish a baseline. Patients were identified retrospectively from patient lists stored on networked computers. The audit standard was 100% completion of all three stages: Sign-In, Time-out and Sign-out. Electronic patient records (EPR) were reviewed to assess the compliance of checklist completion. This also allowed us to assess the compliance of scanning the checklist into electronic record systems – in line with trust policy. Following our intervention of raising awareness via a presentation at a monthly audit meeting on 13 September 2016, we closed the audit loop during October–November 2016. During these periods, we also assessed whether the checklists were scanned in after completion, part of local processes to ensure completeness of EPR.

Data was extracted into a standardized database by the first author (WK). Once complete data was harvested for the two periods, simple descriptive statistics were applied to assess compliance. Ethical approval was deemed unnecessary and not sought given that this is a routine and well established process in this context.

Results

For the period July–August 2016, 70 people met the inclusion criteria. 17 Patients had no scanned checklist, hence a total of 53 patients were included (76%). The mean level of compliance for each phase of the WHO checklist is shown in Table 1 together with the two items with poorest compliance.

Table 1

July–August 2016 Mean compliance and areas with poorest compliance.

	Sign-in (15 items)	Time-out (18 items)	Sign-out (12 items)
Mean Compliance	85%	92%	86%
Top two areas with poor compliance	1. Team Briefing (72%) 2. Does the patient have a safe airway plan in case of difficult airway (89%)	1. Any adverse events anticipated (75%) 2. Specific equipment available if required (79%)	1. Any other issues for follow-up? (21%) 2. Key concerns for recovery (58%)

80 people met the inclusion criteria for October–November 2016. 22 had no scanned checklist; consequently 58 patients were included (73%). Table 2 demonstrates the mean compliance over the second period and the top two areas with poor compliance.

Table 2

October–November 2016 Mean compliance and areas with poorest compliance.

	Sign-in (15 items)	Time-out (18 items)	Sign-out (12 items)
Mean Compliance	86%	98%	89%
Top two areas with poor compliance	1. Team Briefing (82%)	1. Team introduce themselves (93%)	1. Any other issues for follow-up? (38%) 2. Key concerns for recovery (72%)

The overall mean compliance at baseline when all three stages were summated, was 88%, during the second period this increased up to 91% (3.4% increase). The summary of the changes during respective periods is shown below (Fig. 1). The operations that were performed include nail bed repairs, digital nerve and tendon repairs, composite grafts for fingertip injuries, washout and closure of facial lacerations, incision and drainage of paronychia, manipulation under anaesthetic and/or fixation of fractures of the hand. All of these procedures were performed on the emergency theatre list during daytime hours.

Fig. 1

A summary of how compliance changed before and after the intervention for the three parts of the WHO Safer Surgery checklist.

About one in four WHO Safer Surgery checklist were not scanned; 76% at baseline and 73% in October–November 2016. In 22 out of 80 cases, the WHO checklist was not scanned amongst the patients included in the period October–November 2016. This is about one in four checklists. Possible explanations for this matter could be that the scanner in the theatre did not work or new staff were not trained or informed to scan these documents into EPR. In 14 of these 22 cases, the operation note had been scanned, which indicates that the scanner did work. Only 7 of these 22 missing WHO checklists did not have a scanned operation note either, for which we could consider that the scanner did not work and therefore the WHO checklist is missing. In one out of the 22 cases, there were documents scanned to EPR, but these documents could not be opened. Therefore, in most cases not scanning the WHO Safer Surgery checklist was not due to a faulty scanner. However, a spot check on 10th February 2017 showed that 50% of the scanners did not work.

Discussion

Our audit of 150 patients in total over a four-month period, has shown an overall increase in checklist compliance from 88% to 91%. All three stages of the checklist showing improvements, especially the time-out stage, which increased from 92 to 98%. We have also identified areas that consistently carry the poorest level of compliance; in sign-in, the failure to do a team briefing (72% and 82%), in sign-out, any other issues for follow-up? (21% and 38%) and key concerns for recovery (58% and 72%). The overall performance and improvements however, were encouraging.

Throughout both periods, approximately one in four checklists were not scanned. Improving the information technology (IT) infrastructure and support in the hospital could improve the percentage of WHO checklists scanned to EPR. When things go wrong, such documents can be reviewed and potentially lessons learnt. Keeping a record is an important part of our quality assurance system and this feeds into the Trust's indemnity. Scanning important documents like the operation note and WHO checklist is important given the relatively high frequency with which paper medical notes can go missing. Future steps include liaising with the IT department to improve the reliability of the scanner in theatres.

McCulloch et al. recently evaluated 453 operations to assess the impact of interventions designed to improve surgical team performance by culture or systems in isolation or combination [13]. They used compliance with the WHO Safer Surgery checklist as a secondary outcome measure, again highlighting its importance as a safety measure. Epiu et al., conducted a cross-sectional survey on use of the WHO checklist in national referral hospitals in East Africa [14]. Of the 85 anaesthetists interviewed, only 25% regularly use the WHO checklist. The main reason cited for not using it, was because it was not available. The checklist is freely available on the WHO website (http://www.who.int/patientsafety/safesurgery/en/) and a drug reporting similar reductions in morbidity and mortality would be worth many billions annually in global sales. The authors wish to highlight how the most cost-effective interventions can be incredibly simple yet strong compliance with them is still a subject for further study within implementation science. This should not be forgotten as healthcare systems spend precious healthcare budgets on increasingly costly drugs and technology.

There are several limitations to this study. Firstly, that 27% of the checklists were not scanned, so this limits the extent to which we can say the checklist was followed. However assessing compliance with scanning the checklist was one of stated aims and this is highlighted by this work. The other major limitation is how one interprets the form. Lack of documentation does not mean it did not happen. This does also include the relatively high number of operation notes that were not scanned into EPR. They might have been filled out but not been scanned onto EPR. This study focused on paediatric emergency (i.e trauma) patients only, not elective procedures. It would be interesting in a further audit to compare the compliance during elective and emergency surgery. We also only assessed a process measure and not outcomes. Finally, one cannot determine with statistical certainty if improvement has occurred as a result of the intervention we deployed (raising awareness), since only two time periods were assessed.

The pressure on doctors to fill out numerous forms during busy on calls could be reduced by writing operation notes and filling out the WHO Safer Surgery checklist directly on the electronic patient records, which would avoid having to scan paper forms [15]. However, where tablets are not available, having a printed sheet that someone can bring near the patient, which the team can gather around when completing it, can be useful.

Conclusion

This closed loop audit of 150 patients over a four-month period has shown the value of raising awareness of the WHO checklist for optimizing perioperative safety. It identifies ongoing issues of concern to audit against in future, whilst confirming that this process is being done to a high standard overall in the majority of cases with 91% compliance. Further audit is recommended to ensure continued improvement, as well as working with the IT department to improve technology infrastructure to support scanning processes for EPR. Multi-centre audit at the regional and national level may provide further insights into the use of the WHO checklist and help identify common system issues as well as variations in deployment and practice.

Ethical approval

No Ethical Approval was given.

Sources of funding

No funding.

Author contribution

W. W. Keijzer: data collection, writing and approval of the final manuscript for submission.

R. A. Agha: study design, data analysis, writing and approval of the final manuscript for submission.

A. Greig: study design, reviewed the writing and approval of the final manuscript for submission.

Conflicts of interest

None.

Guarantor

W.W. Keijzer, R.A. Agha, A. Greig.

Research registration unique identifying number (UIN)

researchregistry2380.