Increasing use of an endotracheal intubation safety checklist in the emergency department.

Endotracheal intubation (ETI) is a high-risk procedure often performed in the emergency department (ED) in critically unwell patients. The fourth National Audit Project by The Royal College of Anaesthetists found the risk of adverse events is much higher when performing the intervention in this setting compared with a theatre suite, and therefore use of a safety checklist is recommended. This quality improvement project was set in a large teaching hospital in the North West of the UK, where anaesthesia and intensive care clinicians are responsible for performing this procedure. A retrospective baseline audit indicated checklist use was 16.7% of applicable cases. The project aim was to increase the incidence of checklist use in the ED to 90% within a 6-month period. The model for improvement was used as a methodological approach to the problem along with other quality improvement tools, including a driver diagram to generate change ideas. The interventions were targeted at three broad areas: awareness of the checklist and expectation of use, building a favourable view of the benefits of the checklist and increasing the likelihood it would be remembered to use the checklist in the correct moment. After implementation checklist use increased to 84%. In addition, run chart analysis indicated a pattern of nonrandom variation in the form of a shift. This coincided with the period shortly after the beginning of the interventions. The changes were viewed favourably by junior and senior anaesthetists, as well as operating department practitioners and ED staff. Limitations of the project were that some suitable cases were likely missed due to the method of capture and lack of anonymous qualitative feedback on the changes made. Overall, however, it was shown the combination of low-cost interventions made was effective in increasing checklist use when performing emergency ETI in the ED. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Problem

Critically ill patients in the emergency department (ED) often require endotracheal intubation (ETI) via rapid sequence induction of anaesthesia (RSI). This is a high-risk intervention with potential adverse consequences, including inability to oxygenate and subsequent hypoxic organ injury, aspiration of stomach contents, cardiovascular collapse, cardiac arrest and death.1–3 The risk of such sequelae is 30 times greater performing the procedure in ED than in the operating theatre.4 Contributing factors include not identifying potential difficulty prior to the procedure, not assembling the correct equipment, failure to determine backup plans in case of failure and intubation by personnel without sufficient experience.4 5 It has been suggested that the use of an intubation checklist would be beneficial to provide a structured approach to the procedure and to allow cognitive offload in potentially high-stress circumstances.4

This project was undertaken at a teaching hospital in the North West of England providing acute secondary care to a population of approximately 530 000 people, with many living in some of the most socially deprived areas in the country. There are approximately 700 inpatient beds, of which 54 are paediatrics and 24 are adult critical care. Emergency ETI is often undertaken in the critically unwell patient within the eight-bay resuscitation area of the ED. This is nearly always performed by a doctor from the anaesthetics or intensive care unit (ICU) teams, assisted by an operating department practitioner (ODP). Although use of an intubation checklist in this setting is recommended by the Royal College of Anaesthetists4 and is trust policy, it was suspected by the authors that the checklist available was not being widely used.

The aim of this project was for a preintubation safety checklist to be used in 90% of intubation attempts made in the ED by 31 October 2019, which was 6 months from completion of the initial retrospective audit. The model for improvement was determined to be the most appropriate methodological approach.6

Background

Checklists are tools which list action points in a systematic manner and set order. They ensure all items on the list are accounted for and therefore reduce reliance on individual memory, promote standardisation of best practice and reduce error.7 Because of this, they have found utility within high-reliability organisations such those in nuclear power, aerospace, engineering, firefighting and, most famously, aviation.8 In healthcare, the use of checklists to promote safety during high-risk interventions is well established, such as during surgery or central venous catheter insertion.9 10 There are often barriers to checklist implementation and use. Many of these are sociological or cultural in origin. For example, there can be a sense that independence and expertise are not being fully used in deferring to the step-by-step tick-box approach a checklist offers,11 or they can be perceived as simply a tool used to satisfy governance requirements without accounting for the varying complexity of different cases.12 Also, they can be perceived to further complicate situations that are already complex.11

ETI of the critically unwell patient appears an ideal situation for a checklist to be of benefit. Its potential lies in ensuring the correct personnel, drugs, equipment and monitoring are available, and that the patient is physiologically optimised for a successful procedure. Locally, many NHS Trusts within the North West of England promote use of the B@EASE checklist.13 An alternative intubation checklist available is that produced by the Faculty of Intensive Care Medicine, based on the National Safety Standards for Invasive Procedures framework, and they encourage NHS organisations to further modify this to suit local requirements.14 The checklist available for use at our Trust is such a modification. There is also evidence of organisations developing their own checklist15 or modifying template checklists available from national or international organisations.16

No randomised controlled trials (RCTs) are available looking at ETI in the ED setting in non-trauma patients. One preinterventional/postinterventional study showed that use of a checklist in trauma patients consisting of ‘prearrival’ and ‘preintubation’ sections reduced paralysis-to-intubation time and intubation-related complications.17 Another preinterventional/postinterventional quality improvement study performed in the paediatric population showed that a bundle of interventions in the ED had good uptake and increased intubation first-pass success rate without hypoxia or hypotension.18 This included use of a checklist, used in 93% of cases, but it is hard to determine the impact of this individual part of the bundle on the reduced complication rates seen.

A similar non-theatre environment where ETI takes place is the ICU. Despite the presumed advantages, evidence looking at the benefits of a pre-ETI checklist in the ICU is mixed. A reduction in incidence of severe hypoxaemia and cardiovascular collapse has been demonstrated.19 In contrast a multicentre RCT showed no difference in lowest oxygen saturations, systolic lowest blood pressure, procedure duration or number of life-threatening complications.20

Overall, it is assumed there is potential for benefit in using a pre-ETI checklist in the ED and this is recognised by the Royal College of Anaesthetists (RCoA). Therefore, use of a checklist is both a recommendation from the fourth National Audit Project4 and a required standard for Anaesthesia Clinical Services Accreditation (ACSA).21

Measurement

An initial retrospective audit was undertaken to assess incidence of checklist use. All patient records with an ED episode between September 2018 and April 2019 coded with ‘RSI/intubation’ or ‘general anaesthesia’ were reviewed. Coding was undertaken by the ED clinician assessing the patient. The proportion of patients where a preintubation checklist was used was determined by either the presence of a completed checklist or, if this was not present, other documentation stating a checklist was used. It was decided to include the latter as it is assumed the majority of benefit a checklist brings to patient safety is in its real-time use rather than simply acting as a method of documentation. Patients were excluded if only a supraglottic airway device was used; they arrived to the ED already intubated or they were intubated during a cardiac arrest.

In this initial audit, 71 patients were identified. In 10 cases, the relevant episode could not be found within the case notes and these patients were excluded. A further 19 cases met the exclusion criteria stated previously. Of the remaining 42 cases analysed, 3 had a completed checklist with a further 4 having documentation stating a checklist was used without one being present in the notes. Therefore, the total incidence of checklist use was 16.7%.

During the project, patient records were reviewed in the same manner at 3 monthly intervals for the preceding 3-month period. As many patients remained on ICU for prolonged periods, this allowed time for patient episodes to be filed in case notes before review. These audits were approved by our organisation’s audit department. The ED episode coding system changed in April 2019, but equivalent coding items were identified and used to identify cases throughout the remainder of the project. Data were displayed in a run chart and analysed by looking for patterns of non-random variation.22

Design

The core project team consisted of the two authors who were members of medical staff; a CT1 anaesthetics trainee and the RCoA airway lead. The ED resuscitation group was consulted on the problem and their views were sought. This group included medical and nursing staff of various grades involved in improvement of care in the resuscitation area of the ED. As the project continued, further input was sought from an emergency medicine consultant and the lead ED nurse for resuscitation. Patients and other members of the public were not consulted in the design or conduct of this service improvement project as the difference in patient experience between using a checklist or not while in a critically unwell state was estimated to be negligible.

A driver diagram was developed to help determine suitable interventions, and this was structured around three primary drivers (figure 1). These were broad areas considered barriers to use of a checklist. The three areas were first, a lack of awareness that a checklist exists and the expectation by the organisation that it must be used; second, a lack of intrinsic motivation to use a checklist, including not believing a checklist is useful or of benefit; third, not remembering to use the checklist while involved in an appropriate case. Secondary drivers were determined as contributing factors to the primary drivers, and interventions addressing these were developed. It was believed multiple interventions would be required, ultimately addressing secondary drivers linked to all three primary drivers. Of all possible interventions considered, those implemented were the ones estimated to be most effective without being too difficult to undertake. They were trialled using the plan–do–study–act (PDSA) framework.

Figure 1

Project driver diagram. ED, emergency department; ICU, intensive care unit; ODP, operating department practitioner.

There were two points in the project where our opinion of the causes of the problem were shared with others, to see if they agreed or not. The first was on presenting the problem to the hospital’s anaesthetics and intensive care department, and the second was on presenting to the ED resuscitation group. On both occasions, we found the discussions aligned well with what we had already determined around the three key areas which constituted the primary drivers, and we decided not to further alter our approach.

The interventions made were

Increasing awareness of the problem and conveying expectation of checklist use among anaesthetics and ICU medical staff.

Improving visibility and availability of checklists in ED by rationalising those available to the Trust’s own checklist, changing where they were kept as well as considering reminder triggers.

Empowering ODPs and anaesthetic/ICU trainee doctors to use the checklist and to advocate its use to senior staff

Strategy

In the first PDSA cycle, the aim was to increase awareness of the expectation that a checklist should be used and of the existing problem that currently its use was rare. It was hypothesised this was the first barrier to checklist use that needed to be overcome to allow subsequent PDSA cycles a better chance of success. This was done via presentation at the anaesthetic and ICU department audit meeting, attended by 20 consultants, 5 staff grade doctors and 4 trainees. The evidence behind checklist use, their benefits and the importance of complying with guidance from RCoA (within the context of the fourth National Audit Project and ACSA) were discussed. The disappointing results of the initial audit of this project were also shared. After the presentation, the discussion was unanimously in favour of checklist use. Other feedback received indicated that many present at the meeting were unaware of the existence of a checklist and of the expectations around its use. It was also agreed that checklist use is a mandatory requirement. Following the meeting, an email was sent to the entire department to reinforce themes covered in the discussion, reaching 48 consultants, 11 staff grade doctors, 29 trainees and clinical fellows, and 1 critical care practitioner.

The problem was presented in a similar manner to a meeting of the ED resuscitation group. The second PDSA cycle was developed in this meeting, and subsequently by meeting with key ED staff. The aim was to improve accessibility and visibility of a physical checklist in resuscitation bays to act as a reminder in the correct moment to use it. This was believed to be important as even though the first PDSA cycle may have led to increased awareness of the checklist, it could be forgotten about in busy high-acuity situations. There were a mix of Trust and B@EASE checklists available, and these were rationalised to the former. A laminated checklist was attached to the outside of the resuscitation trolleys, which would always be available to be used verbally. In addition, paper checklists were stocked in the airway drawer of the trolley on top of airway equipment. Therefore, they would always be encountered before ETI. The checklist would be added to the equipment list used to restock the trolley to ensure they were always available. Feedback from ED staff was positive as it was believed the changes were quick and simple to undertake while bringing a large potential benefit.

The third PDSA cycle sought to empower ODPs and trainee medical staff to encourage senior staff to use the checklist, as well as use it themselves. This was because most often, the ODP would be the person preparing the airway equipment and therefore encountering the checklist in the resuscitation trolley where they were now kept. An announcement was made to ODPs at their daily meeting that the anaesthetic and ICU departments had agreed that checklist use was mandatory (as per the audit meeting described above, which occurred during the first PDSA cycle). It was emphasised that they should promote use and were reassured that in doing so they would be supported by the department. An email with the same message was sent to all ODPs as it was recognised not all would be at that meeting. This email, with the checklist attached, reached 44 members of staff. Many ODPs indicated they were pleased with this as they generally felt more comfortable in cases where a checklist was used compared with not. Similarly, when new anaesthetic and ICU trainee doctors started their placements, it was announced at departmental induction that checklist use in ED was mandatory. This was done because it was hypothesised new trainees would have arrived from different organisations with varying expectations and practices in this area. Shortly after this PDSA cycle, further outcome measure data were collected.

Results

The outcome measure was the proportion of intubation attempts meeting the criteria where there was evidence of checklist use. Overall, including the baseline audit, 113 patients were identified coded for ETI. Of these, 20 did not have the episode available in the case notes and 26 met the exclusion criteria. The remaining 67 cases were analysed (table 1). This group had a mean age of 50.7 years and 37.3% were female.

Table 1

Identified and analysed cases by month, as well as timing of PDSA cycles performed

Month	September 18	October 18	November 18	December 18	January 19	February 19	March 19	April 19	May 19	June 19	July 19	August 19	September 19	October 19	November 19	Total
Cases Identified (n)	10	11	9	5	12	3	10	11	2	6	6	8	4	14	2	113
Episode missing (n)	2	0	0	1	1	2	2	2	1	0	2	1	0	6	0	20
Meet exclusion criteria (n)	1	4	2	1	6	0	2	3	1	1	0	2	1	2	0	26
Final cases for analysis (n)	7	7	7	3	5	1	6	6	0	5	4	5	3	6	2	67
Checklist present (n)	0	0	1	0	1	0	0	1	0	4	4	3	0	4	1	19
Documented checklist use (n)	0	0	0	0	1	0	2	1	0	0	0	0	2	2	1	9
Checklist used (n)	0	0	1	0	2	0	2	2	0	4	4	3	2	6	2	28
Incidence of checklist use (%)	0	0	14.3	0	40	0	33.3	33.3	–	80	100	60	66.7	100	100	41.8
Month of PDSA cycles (cycle number)									1, 2	3

PDSA, plan–do–study–act.

All data were plotted on a run chart (figure 2). The median was plotted, as was the start of each PDSA cycle. Variation was assessed using established run chart criteria to look for non-random patterns.22 There is a shift beginning at June 2019, coinciding with the time period after the interventions had begun. Before any interventions were made, the overall incidence of checklist use was 16.7%, and after the interventions had begun, the overall incidence was 84.0%.

Figure 2

Run chart showing, by month, the proportion of cases meeting the inclusion criteria where an intubation safety checklist was used. PDSA, plan–do–study–act.

No cases were identified in May 2019 and therefore this data point is missing. More broadly, it was likely applicable cases were missed due to the fact identification for analysis depended on coding by ED clinicians. It is unclear how many such cases there were; however, we are confident the number successfully identified represents a sample sufficient for inferring the impact of the interventions.

Lessons and limitations

This project sought to increase the incidence of safety checklist use before undertaking the potentially high-risk intervention of ETI in the critically unwell. The problem was believed to be one more related to behavioural and decision-making factors than to ineffective steps in a process. Although both areas were addressed in the interventions, a greater focus was on the former through addressing awareness of the checklist, motivation to use it and remembering to use it in the correct moment. We believe this was important to the success of the project and may have been a key reason for the degree of improvement the data suggest occurred, which we had not anticipated. One additional strength of the project was that these interventions required no additional funds, apart from those needed for printing and lamination of additional checklists.

Of the interventions, the most interesting was the presentation at the department audit meeting. Senior colleagues spoke in favour of checklist use both in terms of patient safety and in adhering to required standards, likely creating a degree of social pressure among other colleagues present to use the checklist in future cases. This learning informed the later cycle aimed at ODPs and junior medical staff, but if repeating the project, we would consider how to leverage social influence further to increase use.

This project had a number of limitations. Only cases coded correctly by ED clinicians were included, meaning it was likely a number of applicable cases were not captured. We do not know how many such cases were missed. Further, the coding system changed in April 2019, and this was likely the reason no cases were captured in May. We believe the impact this change of system would have had on the results is likely limited, but we were unable to determine the magnitude of any influence. This did constitute a change in the way episodes of interest were identified and is therefore a significant limitation of the project. More reliable and consistent systems of capturing applicable cases would have been desirable, and if this project were repeated, this is something we would look to establish. Nevertheless, the sample obtained was believed to be sufficiently representative to indicate improvement or not through run chart analysis.

After November 2019, we intended for periodic reaudit to see if sustained use of the checklist had continued beyond what we considered the end of the implementation, for example, at 3 or 6 monthly intervals. At the time of writing, this has not yet been performed. Therefore, our ability to comment on the longer-term sustainability of our interventions is limited.

Information to help assess the impact of any of the individual interventions was limited. For example, in the first PDSA cycle to promote awareness of the checklist requirement, there was no formal prior assessment of existing awareness, or of knowledge areas around checklist use, before the intervention began. Retention of knowledge was also not assessed as the project progressed. This meant quantitative analysis, for example, of a before-and-after nature, of the educational interventions was not performed.

Similarly, the process changes made in the second cycle could have been monitored via a process measure in a longitudinal manner, such as the percentage of cases where the resuscitation trolley had been correctly stocked with checklists or the percentage of cases where a laminated checklist was available. The ability to capture these data was limited by the large number of potential individual staff members performing ETI and the low likelihood most of them would remember to check these during an emergent scenario. These data would have been useful in establishing the impact of those interventions, and if repeated, we would seek to design a way to also monitor this.

A formal investigation of reasons for the initial low incidence of checklist use was not undertaken either. After the driver diagram was created to break down the problem into what we believed to be the major problem areas, discussions were had with the anaesthetics department during the first PDSA cycle and the ED resuscitation group at the start of the second PDSA cycle, as described earlier. Discussions with these two groups served as a check that our initial assumptions around the causes of the problems were correct, but we recognise this is inferior to more formal approaches such as anonymous questionnaires or quantifying contributing factors to the problem via Pareto analysis. This is another area we would explore further if we were to repeat the project.

As far as we are aware, no similar projects were being undertaken simultaneously. While consultants and ODPs are permanent, new trainee medical staff rotate into ICU and anaesthetics departments on a regular basis. The majority of such rotations occur in February and August each year. The run chart analysis showed no changes which would indicate a significant impact on the results, but this does remain a further potential influencing factor.

Assessing and improving how well the checklist was used was outside the scope of the project as it would likely require direct observation of each procedure. Increasing incidence of use was seen as a prerequisite to what would be valuable future work in improving the quality of use in order to maximise the benefit to patient care a checklist would bring. Additionally, assessing the impact of the project on patient outcomes, such as morbidity or mortality, was not within the project scope. This was because we suspected the number of cases analysed would be below the number required to power an analysis on the impact of our interventions on outcomes or a surrogate parameter. This is another limitation of the project. We have made an assumption based on available evidence and recommendations by national bodies that checklist use is of overall benefit to patient care, hence the outcome measure chosen.

We attempted to capture complication rates of ETI to see if our interventions made any impact. Of the 67 included cases, only one complication was documented of vomiting and a drop in oxygen saturations to 85%. It occurred in April 2019 in a case where a checklist was not used. We suspect further complications likely did occur but as the data for this project were retrospective, we were limited to what was documented in each patient’s record. Directly observing the procedures would have provided a more accurate measure of complication rates with or without use of a checklist, but this would not have been possible in our case without significantly more resources. Therefore, we cannot determine the impact of our interventions on complication rates, and this is a further limitation of the project.

Conclusion

ETI in a non-theatre setting such as ED should involve the use of a checklist to minimise the likelihood of adverse events. Our project has shown that as well as changes made to the availability and visibility of the checklists, interventions seeking to positively influence psychological factors involved in deciding to use a checklist can lead to an increase in use. The target of 90% use was set as it was recognised a small number of cases may be so extremely emergent that there would be no opportunity to pause for a checklist run-through, although it could be argued the structured approach a checklist brings is of greater importance in such cases. Although overall 90% was not reached, our data suggest improvement still occurred. It appears use of the checklist has been initially sustained for 3 months beyond the end of the last intervention, and it is likely this was because the interventions sought to engage permanent members of staff, in particular, anaesthetic and ICU consultants, and ODPs. It is yet to be seen whether this will continue on a longer term. Regular reaudit is planned, although this is likely to be at a lower frequency than performed in the project so far. A drop in use seen in such a reaudit would act as a trigger for a further period of intensive improvement efforts. Further interventions generated by the driver diagram, not yet implemented, are available to be trialled if use declines.

We believe similar projects could be performed in other organisations at little cost. Changes to checklist availability and visibility would be relatively easy to make, but determining precisely what is best would depend on the layout of individual EDs and resuscitation bays. Attempting to foster positive attitudes towards a checklist is likely to be more challenging. Our experience in this project suggests it is possible by engaging respected senior colleagues to publicly promote the intervention and to build positive cultural momentum. This is congruent with other evidence that checklist implementation often fails without strong leadership championing it.23 24 Ultimately, such work would need to be tailored to local circumstances and would unlikely be a direct replica of our project.