Patient safety's missing link: using clinical expertise to recognize, respond to and reduce risks at a population level.

INTRODUCTION: Although incident reporting systems are widespread in health care as a strategy to reduce harm to patients, the focus has been on reporting incidents rather than responding to them. Systems containing large numbers of incidents are uniquely placed to raise awareness of, and then characterize and respond to infrequent, but significant risks. The aim of this paper is to outline a framework for the surveillance of such risks, their systematic analysis, and for the development and dissemination of population-based preventive and corrective strategies using clinical and human factors expertise. REQUIREMENTS FOR A POPULATION-LEVEL RESPONSE: The framework outlines four system requirements: to report incidents; to aggregate them; to support and conduct a risk surveillance, review and response process; and to disseminate recommendations. Personnel requirements include a non-hierarchical multidisciplinary team comprising clinicians and subject-matter and human factors experts to provide interpretation and high-level judgement from a range of perspectives. The risk surveillance, review and response process includes searching of large incident and other databases for how and why things have gone wrong, narrative analysis by clinical experts, consultation with the health care sector, and development and pilot testing of corrective strategies. Criteria for deciding which incidents require a population-level response are outlined. DISCUSSION: The incremental cost of a population-based response function is modest compared with the 'reporting' element. Combining clinical and human factors expertise and a systematic approach underpins the creation of credible risk identification processes and the development of preventive and corrective strategies.

‘While it seems obvious that collecting data and not analyzing it is of little value, the most common failure of governmentally run reporting systems is to require reporting but not to provide the resources needed to analyse the reports. Huge numbers of reports are collected only to sit in boxes or on computers. Expertise is a major, and essential, resource requirement for any reporting system …

Ultimately, it is human experts who must translate the knowledge gleaned from aggregated reports into meaningful recommendations for action to improve care.’ World Health Organisation, 2005

Introduction

A high rate of health care-associated harm is a consistent finding across health systems in developed and developing countries [1-3]. Many governments, health departments and private organizations have implemented incident reporting systems (IRSs) across networks of their health care facilities [4, 5] as part of a strategy to reduce this harm. Investment in these systems has been considerable [6, 7], but the focus, at a population level, has been on reporting problems rather than responding to them [7-9]. Learning from and responding to things that go wrong has also been lacking for collections of complaints, medico-legal files and coroners' recommendations [10, 11].

Widespread acceptance of incident reporting and improvements in the ease of reporting with web-based IRSs has resulted in some systems collecting large numbers of incidents, albeit mainly from hospitals. For example, annually >1.5 million incidents are collected in England and Wales [12], 200 000 in the US state of Pennsylvania [13] and 100 000 in the Australian state of New South Wales [14]. Satisfactory processes for local responses to individual incidents have been developed in many organizations [4]. However, systems to realize the potential for gaining a rich understanding of the full range of things that go wrong at a population level are largely undeveloped [8].

Notwithstanding considerable progress in the use of additional methods, such as case note review [15, 16], automated data extraction [17], observational and ethnographic studies [18, 19] and the routine collection of safety metrics [20], to characterize the nature and antecedents of common types of incidents, there are still untapped benefits of large collections of incidents. These retain a unique capacity to identify and understand infrequently occurring patient safety risks, which are unlikely to be characterized at a hospital or local level [21]. They can also provide early warnings of the inevitable yet unforeseen new risks associated with changes in health care practices and the introduction of new technologies [22, 23]. Indeed, analyses of large collections, together with other sets of information about things that have gone wrong, are likely to be the only way to gain insight into these risks [21]. Although clinical case reports have a place in promoting debate on rare events, large collections allow identification of common contributing factors or repeated patterns of error by collating incidents from multiple institutions [4, 11] (Boxes 1–3 [24-26]). However, being infrequent does not mean being insignificant, as these types of incidents may result in substantial harm to patients at a population level [21]. There is, thus, a strong case for developing preventive and corrective strategies for low-frequency risks at regional or national level.

Adult urinary catheters are manufactured in two sets of lengths: female (20–26 cm) and male (40–45 cm). Mis-selection of a female length for a male can cause urethral trauma and haemorrhage when the retention balloon is inflated. The National Patient Safety Agency (NPSA) located 114 of such incidents and reviewed them with a view to developing national corrective strategies. Most incident reports did not provide sufficient information on contributing factors. It took clinical insight into anatomy, indicators for catheterization and labelling and storage on a typical hospital ward to identify this risk that could be addressed through controls on how catheters were labelled and stocked.

Nasogastric feeding tubes are commonly used for patients with dysphagia or those on ventilators. Such patients can be harmed if the tube is mistakenly inserted, or later becomes displaced, into a bronchus. The NPSA released an alert in 2005 [25] on the need to confirm correct placement. However, the NRLS subsequently received reports of 21 deaths and 79 cases of harm due to lung contamination from feeding fluid into bronchi, mainly associated with misinterpretation of confirmatory X rays (12 deaths and 45 cases of harm). The focus of a new Alert released in 2011 supports safe X-ray interpretation. Other factors contributing to harm were: feeding despite obtaining aspirate with pH between 6 and 8 (two deaths, seven incidents), instilling water down the tube before aspirating (two incidents) and no check of tube placement at all (one death, nine incidents). A repeated finding was no written record of pH of aspirate or X-ray confirmation of placement.

The recommendations included having a named clinical lead, auditing compliance, assessing staff competencies, the purchase of radio-opaque nasogastric tubes and avoiding placement of the tubes ‘out-of-hours’ unless urgent.

In 2010, the NPSA received a number of concerns from anaesthetists, risk managers and pharmacists relating to incidents of inadvertent intravenous paracetamol overdose in children. A search of the NRLS identified 206 relevant incidents associated with 44 neonates and 162 children resulting in two cases of severe harm, 14 cases of moderate harms and the remainder of low or no harm.

Incident themes included:

lack of awareness amongst health care professionals of neonatal and paediatric drug dosage regimens;

patients receiving doses of paracetamol in both theatre and wards due to poor documentation;

human error relating to setting up infusion pumps to administer intravenous paracetamol (incorrect rates or volumes to be infused);

confusion between dosing regimens with clinical staff believing doses for oral and intravenous paracetamol are interchangeable;

ten times dose calculation errors in both the prescription and administration of intravenous paracetamol.

The NPSA did not issue new recommendations, but referred to the MHRA Drug Safety Bulletin, released in July 2010. The Bulletin advised that dose should be based on weight and provided recommended dosing regimens. It also advised that for infants and children weighing <33 kg, the 50 ml vial should be used for administration.

In this paper, we outline a framework for using clinical expertise to recognize, respond to and reduce risks at a population level. This has been informed by the experiences of the authors in managing large-scale IRSs in Australia, the United Kingdom and North America. The framework comprises system and personnel requirements and a multistaged risk surveillance, review and response process.

Requirements for a population-level response function

System requirements

There are four system requirements. First, a means for reporting incidents that is available to all health care workers and is adequately resourced, non-punitive, independent and confidential [8, 10, 27]. Second, software to aggregate incidents and other information from multiple sources and institutions. This may directly collect information in one database (e.g. Pennsylvania [13], New South Wales [6]) or upload it from independent systems into a central repository (such as the National Reporting and Learning System (NRLS) in England and Wales). Third, a system for the production and management of reports for each stage of the risk surveillance, review and response process (Fig. 1); this may be facilitated by workflow and document management software. Fourth, a system to disseminate corrective strategies to health care organizations is essential (see Fig. 1, Stage 8).

Figure 1

Stages of the population-level risk surveillance, review and response process.

Personnel requirements

A multidisciplinary team comprising clinicians, subject experts (e.g. pharmacists and biomedical engineers) and human factor experts should be assembled to advise and manage the risk surveillance, review and response process (Fig. 1). The meaning of many incident reports can be gleaned only through interpretation and the application of high levels of judgement from a range of perspectives (Box 1). Clinicians' understanding of typical workflows and health care organizations' operations play an important role in their ability to interpret incidents [28, 29]. Subject-matter experts assist in understanding the patterns of contributing and contextual factors [29]. Human factors personnel can advise on common error mechanisms [30] and the development of corrective strategies which are ‘strong’ [31] and sustainable.

Personnel with a range of experience and specialties should be included in discussions that are open and non-hierarchical. For example, a wrong dose error in a neonatal intensive care unit will prompt different responses from a pharmacist, a neonatal nurse, a junior doctor and a human factors expert. Only by seeing the story through multiple lenses can the true nature of the systems weaknesses be revealed [32]. Subjecting the composition of the team and interim conclusions to peer review and seeking constructive challenge are also desirable.

Risk surveillance, review and response process

We have outlined nine stages of a risk surveillance, review and response process (‘the review process’) (Fig. 1); however, not all of these are necessary for some types of risks. For urgent risks, the entire process can be completed within a week. The multidisciplinary team can provide expertise, as necessary, at each stage. An example of a risk managed through these stages is shown in Box 4 [33-37].

Stage 1: Undertake surveillance. All incidents reported and uploaded to the NRLS within the previous month and classified as resulting in ‘severe harm’ or ‘death’ are extracted into the incident response management system.

Stage 2: Identify a ‘trigger’ incident. A Clinical Reviewer read the following report ‘Finger tourniquet left in situ for 14 days following minor surgery for wound debridement pulp left middle finger. Patient required amputation of finger. Initial operation performed on day one, tourniquet discovered on day 13 and amputation of the left middle finger carried out on day 14’ and thought that this incident may meet the criteria for a national response. It was referred to a multidisciplinary team meeting.

Stage 3: Collect like-incidents. A key word search of the NRLS database found 149 cases. Narrative review confirmed 15 relevant incidents, six of which involved surgical gloves being used as tourniquets. The NHS Litigation Authority also had 14 relevant claims with the highest payment exceeding £100 000. A literature search found only two case studies reporting harm from tourniquets left on in error, but a paper from the Pennsylvania Patient Safety Authority in 2005 described 125 reports from the previous year.

Stage 4: Characterize the relevant incident type. Tourniquets are used in hand and foot surgery to obtain a bloodless field, often for minor surgery in settings such as emergency departments, GP surgeries and podiatry clinics. Flesh-coloured surgical gloves are often used as they are freely available and are cheaper than tourniquets. There are no data on the relative safety of tourniquets and other devices (such as gloves). However, purpose-designed visible tourniquets are available and data from reported incidents suggest that at least some of the preventable harm is caused by the use of surgical gloves. Problems of visibility with tan-coloured tourniquets and the vulnerability of patients at the extremes of age were noted by Pennsylvania Patient Safety Authority [34] and human factors experts.

Stage 5. Identify preventive and corrective strategies. Advice was sought from the Royal College of Surgeons and others, and some key aspects of safer practice were identified including (i) controlling/reconciling the number of tourniquets used via checklists, (ii) using purposely designed tourniquets, (iii) using tourniquets with high visibility design features, including labels and colour and (iv) informing patients and/or family regarding the use of digital tourniquets.

Stage 6: Develop materials for a response. An action-based ‘rapid response report’ (RRR) was decided to be the most suitable mechanism for a response. More detailed evidence on the nature of the problem and a literature review was developed as ‘Supporting Information’ [35]. A clinical briefing [36] and a BMJ safety summary [33] were also released.

Stage 7: Pilot test and refine. Deemed not necessary for this problem.

Stage 8: Disseminate alerts and corrective strategies. The one-page RRR [37] was released incorporating a description of the issue, the scale of the problem and corrective strategies:

Guidelines include the removal of digital tourniquets as part of the swab counting procedure and specify the need to record the length of time a tourniquet is in place.

Digital tourniquets which are labelled and/or brightly coloured should be used, in accordance with manufacturers' instructions. Surgical gloves should not be used as tourniquets.

The WHO Surgical Safety Checklist should be reviewed locally to consider adding tourniquet removal at the ‘Sign Out’ stage.

The NPSA clinical briefing sheet should be used to raise awareness of risks using digital tourniquets and safer practice recommendations.

Stage 9: Evaluation. Early information from the manufacturers producing tourniquets shows a 140% increase in purchasing in the three months after the issue of the RRR compared with a similar period before issue [33]. Three months post-alert release, no further incidents of harm from tourniquets left on after finger or toe surgery had been reported to the NRLS.

Stage 1: Undertake surveillance. Categorical or free-text algorithms can extract those incidents more likely to be candidates for a national (or population-based) response from the incident database. Given that tens or hundreds or thousands of incidents may be reported each year, those associated with serious harm may be prioritized. Other sources, such as media, coroner's and medico-legal reports may also be scanned for potential serious risks.

Stage 2: Identify a ‘trigger’ incident. Narratives should be read by an experienced set of clinicians to determine if a ‘trigger’ incident or cluster of incidents merits a population-level response. Criteria to determine which incidents, or clusters, should qualify are outlined in Box 5 and have evolved through experience; these criteria should be considered at each stage of the review process.

A new or under-recognized risk:

which is ‘novel’, or

whilst not ‘novel’, is not well known or has not been given the priority it merits, or

involves new technologies or health care processes, or

is part of a pattern or trend of similar but previously unrecognized incidents

AND

There is evidence of actual or potential significant patient harm (severity of outcome and frequency).

AND

Preventive and corrective actions are feasible, are not already widespread and may be implemented in a cost-effective manner.

Stage 3: Collect like-incidents. For trigger incidents which may meet the population-level response criteria, searches of the database should be conducted for similar events. The aim is for estimates to be made of the frequency and severity of the consequences of the type of incident in question. If there is evidence of significant patient harm, the literature and other available sources can be searched for similar incidents. Opinions and anecdotes may also be sought from experienced clinicians and administrators.

Stage 4: Characterize the relevant incident type. The collected incidents should be subjected to categorical and free-text analyses, including iterative theme-based analyses of the narratives. The categories in the International Classification for Patient Safety (ICPS) [38] (e.g. contributing factors, method of detection and mitigating factors) can provide a framework for such an analysis. A thorough understanding of themes arising from the analysis may suggest which categories of the ICPS should be targeted for preventive and/or corrective strategies (e.g. better detection, or attention to a contributing factor, such as staffing).

Stage 5. Identify preventive and corrective strategies. Preventive and corrective strategies should be investigated to identify steps that could be taken which are acceptable, feasible and affordable, and which are not likely to result in unintended consequences. All available information should be taken into account, including details identified in previous stages and during consultations with clinical specialists and expert organizations such as medical colleges. The ‘strength’ of draft preventive and corrective strategies can be also evaluated [31].

Stage 6: Develop materials for a response. This may be an ‘alert’ to draw the attention of health care providers to new or under-recognized risks, or risks that are not being adequately managed, and may prescribe or suggest corrective strategies. A tiered response such as that used by NHS England (Box 6) [39] may be appropriate. Examples of alerts with and without corrective strategies are shown in Boxes 2 and 3, respectively. The tiered definitions [39] and response formats (Box 7 [33, 40–42]) need to be flexible enough to accommodate problems with very low-frequency and severe outcomes (such as intrathecal administration of vincristine) [43], as well as those occurring at higher frequencies but with generally less severe outcomes (such as omitted medications) [44].

Stage One Alert: Warning

This stage informs organizations of emerging risk. It can be issued very quickly once a new risk has been identified to allow rapid dissemination of information.

Stage Two Alert: Resource

This alert may be issued some weeks or months after the stage one alert and would notify organizations of resources that might include: Stage Three Alert: Directive

sharing of examples of local good practice that mitigates the risk identified in the stage one alert;

access to tools and resources that help providers implement solutions to the stage one alert; and

access to learning resources that are relevant to all health care workers and can be used as evidence of continued professional development.

When this stage of alert is issued, organizations will be required to confirm they have implemented specific solutions or required actions within a set timeframe. These actions will be tailored to the patient safety risk and could include actions that are not feasible for organizations acting alone (e.g. a standardization of a clinical process).

As brief as possible, a one-page summary, and, if necessary referring to expanded notes on subsequent pages:

A succinct statement of the patient safety risk

The departments or health care provider types, or health care staff to whom the patient safety risk is relevant

The evidence for the risk including the number of incidents and the harm that resulted. Although some organizations may consider this information politically sensitive, it is vital for clinicians and health care managers to understand the level of patient harm

Examples of incident narratives that illustrate the risk

Recommendations (corrective strategies) that will reduce risks to patients—these should be clear and auditable (or ‘SMART’ (Specific, Measurable, Acceptable, Responsive, Timely) [40])

When corrective action needs to be taken or completed by.

Other supporting information may supplement the core one-page summary such as evidence summaries from the incident data that prompted the corrective strategies.

Resources to support implementation including compliance checklists, good practice examples, patient information resources, visual aids and posters. Additional publications may be produced aimed at getting the attention of target audiences (particularly clinicians), such as ‘one-liners’ for clinical directors or doctor-friendly bulletins in medical journals [33, 41, 42].

Stage 7: Pilot test and refine. The materials and strategies developed should be tested in pilot studies at a number of health care facilities. Managers and key staff ‘on the ground’ need to be encouraged to raise practical concerns and consider cost and service implications. Lessons learned should be fed back to the multidisciplinary population-level response team and used to refine strategies for dissemination.

Stage 8: Disseminate alerts and corrective strategies. Many health departments and governments have existing systems to disseminate recommendations related to medicines and equipment regulation to health care organizations. These are generally used for recall or action when a medication or equipment safety issue has been identified. If these systems are set up with the necessary roles and permissions, they can also be used to disseminate the corrective strategies derived from responses to risks more broadly. NHS England's Central Alerting System [45] provides an example of such a system. The general characteristics of these systems are: mandatory participation by all health services; the ability to broadcast advice to all health services; a named person at each health service being responsible for receipt of advice and following up recommendations; health services being able to view and follow their progress; and the production of status reports. As well as the official dissemination channel, additional mechanisms for communication are recommended including emails to individuals, press releases and information sheets or articles via professional organizations.

For responses to risks which do not meet the threshold for corrective strategies to be developed and disseminated, there are other options. Information about specialty-specific risks can be released via professional bodies such as colleges or societies or published in specialist journals to target a relevant readership.

Stage 9: Evaluation. In this stage, the aim is to determine the effectiveness of the population-level response function including risk prioritization and characterization and corrective strategy dissemination and implementation. Data sources and methods will depend on the strategies being evaluated and could include clinical process audits, equipment and medicines procurement record audits, and surveys or interviews of clinical and administrative managers responsible for actioning the recommendations. However, it is important to note that evaluation is seldom straightforward [4, 7]. Incident reporting rates should not be used as a measure or subjected to statistical analysis as they vary considerably over time, and within and between institutions [7]. There are no denominators and what is reported is subject to considerable bias [46]. Indeed, reporting rates may increase following recommendations as awareness of the problem improves. For example, in the period immediately after the alert on mis-selection of female urinary catheters for use in males described in Box 1 [24], incidents which were previously seen as unexplained pain and haematuria were correctly recognized and reported.

Analysis, or counting, of outcomes is also problematic. Many outcomes, such as wrong-site surgery, are too rare to find statistically significant trends in rates [21], and attributing a change in outcomes to the release of an alert, or an intervention, is difficult due to the complex nature of health care [11]. Proxy process measures have been used. For example, 1 year after an alert recommending a lower dose of midazolam, there was a 2- to 3-fold increase in purchasing the lower dose [41], and 3 months after the recommendations in Box 4, there was a 140% increase in purchasing of the digital tourniquets [33]. Surveys may also be conducted. For example, in Pennsylvania, Patient Safety Officers are surveyed annually to identify recommendations that led to changes in their facility. Hundreds of examples of changes to policies and clinical processes that positively affect safety are received from the Patient Safety Officers [47].

Discussion

We have presented an approach to recognizing, prioritizing, characterizing and responding to low-frequency patient safety risks in large collections of incidents. The aim is to disseminate preventive and corrective strategies which are proportionate, achievable, cost-effective and acceptable to the clinical community and recipients of health care. Such a function is designed to complement, rather than replace, the functions of local incident reporting which are fundamental components of clinical governance by providing mechanisms for a local response to identified risks as part of a ‘duty of care’ [10].

A call for a system emulating that of the Commercial Aviation Safety Team, incorporating multiple organizations and skill sets, confidentiality protections and designing and implementing strong corrective strategies has been made by prominent proponents of patient safety [48]. The principles we have outlined can be adopted with a sufficient number of incidents at a state or provincial level, such as New South Wales or Pennsylvania, or across large private hospital organizations. Indeed, a multinational response function is also possible and may be desirable for countries with smaller populations and incident databases. This may also be suitable for specialty collections, for example, anaesthesia, radiology and radiation therapy, where incident numbers may be lower, but still substantial.

Decisions relating to incident frequency, severity and clinical relevance are often based on incomplete information and characterized by a high level of uncertainty, even amongst experienced clinicians [11]. For example, it is difficult to compare societal impact, financial cost and level of preventability of 20 000 incidents involving omitted medicines [44] and resulting in a variety of levels of harm, with the loss of 15 digits from gangrene caused by tourniquets [33]. Although we advocate a standardized system for priority setting, the variety and number of risks in a complex system such as health care means that consistent decision-making is challenging. This strongly emphasizes the importance of involving clinicians from a wide range of specialties and the incorporation of human factors principles [50-52]. This is analogous to the approach in the aviation industry, where priority is given to the interpretative work of highly skilled analysts in understanding and learning from, and responding to, errors [53].

There are well-founded criticisms about incident reporting not being representative of the harm that occurs in health care [49] as well as incompleteness of reports raising queries about data quality. Having access to other data sources to triangulate information derived from incident reports may mitigate some of these concerns by providing a more complete profile [50]. However, the strength of voluntary reporting is that incidents have been prioritized by busy clinicians; this may, in itself, positively skew reported incidents towards those where remedial action is most likely to be needed or feasible. Additionally, in contrast to other data sources, incident reports may contain information on contextual and contributing factors that are valuable for analysis and corrective strategy development [11, 51].

IRSs require considerable investment [6] by way of software and hardware infrastructure, IT support and staff time to report and manage incidents locally. If this investment exists in isolation from an effective population-level response function, IRSs are open to valid criticisms with respect to their value and purpose [52]. A small, skilled clinically led response team using a structured review process, representing a marginal additional resource, can improve efficiency by developing corrective strategies just once at a population level.

Challenges

Challenges to operating a population-level response function include:

Detecting and responding to new or under-recognized threats to patient safety must be balanced against undertaking prospective programmes such as quality improvement initiatives to address well-known risks. Both approaches are valuable and complementary. Indeed, these population-level response functions can still inform common incident types. For example, even though the NRLS has collected over two million incidents describing inpatient falls, and approximately one million medication incidents [12], new and under-recognized risks continue to emerge and may be the subject of alerts [54, 55].

Determining the threshold for issuing new directives and defining the number which should be released every year. The capacity of the health care system to respond to recommendations is finite, with competing demands from local IRSs and complaints, medication, equipment and blood product regulators and health-care accreditation and regulation agencies.

Ensuring that channels within healthcare organizations to disseminate relevant safety information to clinicians are adequate. For example, in England, around half of NHS trusts cannot communicate effectively and reliably with their junior doctors [56] posing challenges to inform them of relevant safety risks.

Conclusion

Significant resources have been invested in IRSs, and the value of these has been challenged. We have outlined the key features of a population-level function to recognize and respond to risks using clinical expertise. The incremental cost of such a function is modest compared with the cost of deploying the ‘reporting’ element. Combining both clinical and human factors analyses at the heart of the response is likely to create credible and acceptable risk identification processes and preventive and corrective strategies.

Whether a national function to respond to patient safety risks is necessary can be answered with the following questions: if, at a hospital in your country, a tourniquet was left on a patient's post-surgical finger—causing gangrene from ischaemia necessitating amputation, would other hospitals be informed, and how would this occur? Would recommendations be developed to reduce the incident happening again to another patient in your country? Would hospitals be under any obligation to implement the recommendations? Or, how would clinical staff, patients and their relatives feel if they discovered that many similar incidents had been reported, but no-one had acted on these to alert other hospitals?

Funding

This work was supported by National Health and Medical Research Council Grant No. [1054146].