Revised Risk Priority Number in Failure Mode and Effects Analysis Model from the Perspective of Healthcare System.

BACKGROUND: Methodology of Failure Mode and Effects Analysis (FMEA) is known as an important risk assessment tool and accreditation requirement by many organizations. For prioritizing failures, the index of "risk priority number (RPN)" is used, especially for its ease and subjective evaluations of occurrence, the severity and the detectability of each failure. In this study, we have tried to apply FMEA model more compatible with health-care systems by redefining RPN index to be closer to reality.
METHODS: We used a quantitative and qualitative approach in this research. In the qualitative domain, focused groups discussion was used to collect data. A quantitative approach was used to calculate RPN score.
RESULTS: We have studied patient's journey in surgery ward from holding area to the operating room. The highest priority failures determined based on (1) defining inclusion criteria as severity of incident (clinical effect, claim consequence, waste of time and financial loss), occurrence of incident (time - unit occurrence and degree of exposure to risk) and preventability (degree of preventability and defensive barriers) then, (2) risks priority criteria quantified by using RPN index (361 for the highest rate failure). The ability of improved RPN scores reassessed by root cause analysis showed some variations.
CONCLUSIONS: We concluded that standard criteria should be developed inconsistent with clinical linguistic and special scientific fields. Therefore, cooperation and partnership of technical and clinical groups are necessary to modify these models.

Introduction

Many types of research which used risk management models were conducted to improve defects of risk assessment in the health-care environment.[1] Failure Mode and Effects Analysis (FMEA) is an industrial biased model with the ability of systematic assessment of a very complex process.[234] Several clinical disciplines applied the model successfully such as blood transfusion,[5] diagnostic radiology,[6] and medication prescription.[7] However, low reliability and validity the of FMEA have been challenged in the literature.[8] Several studies have been carried out over the past decade to improve the FMEA outcomes and the limitations of risk priority number (RPN). Conventional FMEA assesses the occurrence of failures appertaining to the experiences of practitioners and agreements of the team,[7] but clinical terminology has not been used in it.[910] Disadvantages of the conventional FMEA in evaluating the risk brought about suggestions on several risk priority models to prioritize the failure modes. Considering the fact that three factors of severity (S), occurrence (O) and detection (D) have different weights is important in risk assessments. Logically, the significance of S and O factors are more than D factor for some irreparable systems. In addition, different combinations of O, S, and D may computationally create the same values of RPN.[1112] Many calculation problems in conventional FMEA depend on lack of accurately quantified values and trust just in experience of practitioners.[310]

We believed that RPN index can be reassessed with root cause analysis (RCA) model. RCA is a retrospective model that requires health-care professionals report deviations from normal practice. For prioritizing risk of events, the severity of their consequences and the likelihood of their recurrence is then calculated.[13] In this study, we investigated the failures using FMEA and reassessed reliability of FMEA by RCA assessment. This study introduces an adapted method for solving challenges of the FMEA model in the health-care system. Capabilities are enhanced; in that industrial model make more applicable in health-care sector through revision of RPN. This is obtained by integrating FMEA and RCA, a multi-criteria decision-making technique. Although this model is recommended by the JCAHO as one of risk assessment procedures,[14] there are only scarce reports on its application in surgery.[15] In addition, surgical adverse events (AEs) consists 51%−79% of all AEs related to surgical wards and 43% of these AEs are preventable.[1617] In this study, we evaluated the model in the surgery ward.

Methods

This study employed a qualitative and quantitative approach. Procedure determined in two main steps:

First, an initial framework developed through meetings with health management professors and literature review to identify contributing factors and effects of surgical AEs. Then, a content analysis was performed to extract key concepts and sub-concepts. The concepts were preventability, hospital stay, complaint, harm to patients, out-patient care, cost, and frequency of events

Then, the revised RPN was implemented and verified in a pilot study on patient journey process to evaluate its applicability and reasonability:

During the focused groups discussion (FGD) meetings, steps in the patient care in general surgery ward were prioritized based on their role in patient safety enhancement and risk reduction of nosocomial events and a researcher-made checklist (degree of importance of each phase was determined on a scale of 1–5).

Then, failure modes of the prioritized phase were determined according to FGD opinion in brainstorming meeting and interviewing. Next step was to determine the important concepts of RPN criteria in the health-care system. The method of calculating severity, occurrence, and detectability scores (DSs) were determined by quantifying selected criteria. Then, RPN of each failure was calculated with newly developed criteria. As the final step, we followed up AEs with higher RPN score for 3 months to benchmark ability of improved RPN to prioritize risks correctly [Figure 1].

Figure 1

Flowchart of research methodology steps

An inductive approach was used to extract RPN concepts. A purposeful and stratified sampling of key informants was performed. Members of the FGD included head of clinical governance unit, director of accreditation unit, head of operating rooms; head of department anesthesiology, recovery head nurse, head of day clinic (one of department in the hospital for admitting nonhospitalized patients before surgery), two general surgeons, and two nurses from the operating room and recovery room. FGD meeting conducted in 8 sections of 3 h. For each phase of the process, two sections were required.

RPN was calculated the severity of event (S), the probability of occurrence (O) and probability of detection (D) according to the following formula: RPN = S × O × D. The RPN value for each failure ranges between 1 and 1000. An acceptance limitation was set for RPN score based on the previous studies. RPN of more than 300 was considered unsafe.

Results

Phase 1: Failures of the highest priority phase

Steps in patient journey process in surgery ward were identified with regard to the SURgical PAtient Safety System checklist.[18] The most important phase identified to be phase 3–1 (see below). It was the nearest step to the main phase of the operation and hence more chance of preventing AEs or sentinel events as the last defensive barriers (DBs).

Preoperative care of the surgical patient before entering to the surgical ward

Preadmission clinical assessment before entering the holding area of the surgical ward

Patient flow from the holding area to the operating room

Transfer of patient to anesthetic care

Transfer of patient to recovery bay

Transfer of patient from recovery bay to Intensive Care Unit or related ward.

Finally, 12 failure modes determined for 3-1 phase:

The displacement of the operation plan or patient's document

Failure to correctly identify patient by surgical team

Substitution of companies' surgical sets with surgery ward instruments and sets

Incorrect enumerating and keeping track of swaps and sponges, needles and other retained instruments and supplies

Unsafe use of cautery devices

Missing pathological samples

Inappropriate or incomplete surgical sets provided for surgery

Surgical instrument sets or prosthesis purchased mismatched with surgeon's order

Not having preparedness in providing sufficient surgical instrument sets, packs, consumable and nonconsumable instrument and supplies

Failure to provide correct operation report sheet and physician orders sheet

Miscounting of radiographies, ultrasound, computed tomography, and magnetic resonance imaging before delivering patients to anesthesia team

Not cleaning blood and purulence on patient's body, not separating all connections and cover patient appropriately

Positioning patients without informing the anesthesia team.

Phase 2: Determining inclusion criteria for priority failures

After reviewing literature and FGD meetings, themes related to risk priority criteria in surgery ward was extracted. These themes included preventability and DBs in the field of detectability,[1619] patient deaths, extra intervention or treatments, disability at discharge, readmissions, prolonged hospital stay, outpatient visits, physical injury, mental distress, pain, suboptimal care and inconvenience in the field of clinical injury,[1620] patient satisfaction and medico-legal complaints,[2122] cost and time[23] and finally risk exposure and risk frequency within a certain period in the field of occurrence.[2425]

Phase 3: Quantifying risks priority criteria in the context of risk priority number in Failure Mode and Effects Analysis model

FGD members suggested that a reliable and valid measurement tool be used to avoid bias. Hence, quantifying inclusion criteria inconsistent with clinical linguistic was used for this purpose.

Severity criterion

First, the scales of clinical injury, medico-legal consequences and wasted time and costs were identified in six levels from low priority to a high priority for each tree sub-criterion. Then, the highest weight attributed to the clinical injury (W1= 3) and the least to legal consequences (W2= 1). The significant coefficient (SC) determined for each level to calculate severity score (SS) from 1 to 10 [Table 1].

Table 1

Occurrance sub-criterion weight and scales

Table 2 shows severity sub-criterion weight and SC.

Table 2

Severity sub-criterion weight and significant coefficient

Weight (Wj) = Weight of the sub-criterion biased on its importance

SC = SC allocated to each row of Table 2

SS = Total score of the severity criterion [Table 1]

SS maximum = SSm = 10

SL = Severity levels based on importance of sub-criterion negative impact

*SL maximum = SLm = 6

1 ≤ SL ≤ 6 s

1 ≤ SS ≤ 10 (standard severity number according to the FMEA model)

SC of sub-criteria scales multiplied by weight of each column (formula number 1)

Formula number 1:

Formula number 2:

Occurrence criterion per unit of time and risk exposure was suggested to occur for every 1500 patient exposed to the failure modes. That is because each group of patients would be exposed to a certain number of failure modes according to their planned procedures. Hence, nature of failures considered as an important factor to calculate denominator and numerator [Table 1]. Score 10 indicates the highest occurrence possibility and score 1 indicates the lowest one.

If the scales of occurrence per unit of time (OPT) and risk exposure ratio (RER) were not considered in Table 3, the interpolation formula would be used (formula number 3).

Table 3

Detectability sub-criterion weight and significant coefficient

Formula number 3

Table 1 shows occurrence sub-criterion weight and scales

OS = Occurrence score (standard OS according to the FMEA model considered: 1 ≤ OS ≤ 10)

RERi2 = RER in i level

OPTi1 = OPT in i level.

Formula number 4.

Detectability criterion

Two sub-criteria of preventability and DBs have been given equal weights. SC determined for levels of each table to calculate DSs between 1 and 10 (formula number 6) [Table 4].

Table 4

Severity of failures, frequency of occurrence and detectability of failure modes of selected phase in patient's journey process for surgery

Table 3 shows detectability sub-criterion (DS) weight and SC

DS = DS (standard OS according to the FMEA model considered: 1 ≤ DS ≤ 10)

DSMax = DSm = 10

DLMax = DLm = 5

1 ≤ DL ≤ 5.

Formula number 5

Formula number 6.

Table 4 shows severity of failures, frequency of occurrence and detectability of failure modes of selected phase in patient's journey process for surgery

Phase 4: Benchmark ability of improved risk priority number to prioritize risks correctly after 3 months

Among the 12 failures reported, 11 failures were related to the 3-1 phase which supports our proposition that 3-1 phase is the most important phase of the patient journey. Furthermore, the most frequent events with highest RPN score were related to the patient misidentification during last 3 months. On the other hand, clinical injury, legal consequences, wasted time and costs, DBs and preventability in three events differed with what was predicted in the RPN.

Table 5 shows predicted prioritization criteria for Failure No 2 (Patient misidentification) against real AE/NM RPN after 3 months.

Table 5

Predicted prioritization criteria for failure 2 (patient identification mistakes) against real adverse events/NM risk priority number after 3 months

As similar blood group transfuse accidentally and unwittingly, the real scale of both clinical injury and wasted time and costs decreased to lower priority. However, it is important to consider that this AE could be more harmful to the patient if different blood groups transfused during surgery

Since sex identification of the patient had been mistaken and gender didn't have any effect on surgery, there would be no threat to the patient

Although standard form and Electronic Healthcare incident reporting system have been setting up, there was no document or evidence about reporting AE/SE/NM. Even head nurses and director of wards would not be informed orally. So, we could not estimate the real scores of RER OPT in all three AEs/NM cases.

Discussion

In this study, we developed an improved version of S, O and D main concepts through analysis of key informants' viewpoints to make the analyses situations more accurate. Then, concepts made measurable to predict the most possible failures and reassess accuracy by RCA after 3 months. The most important phase shown to be the transport of the patient from the holding bay to the operating room. The most frequent AEs occurred was patient misidentification after 3 months. Misidentification score of S, O, and D was somehow different with what was predicted in sub-criterion.

In Yeh study, the fuzzy FMEA approach was used for transforming the process to avoid the shortcomings of the conventional RPN. Results of empiric validation indicated that fuzzy theory made RPN more significant.[10] Two main deficiencies of the conventional RPN index include (1) different scales of severity, occurrence and detection criteria may produce same RPN values and (2) calculating unreal numerical values when the team disagrees in scoring the criteria. Sellappan and Palanikumar introduced a new RPN method to overcome these deficiencies. The proposed method has been audited by doing case studies and statistical analysis (1–10). Esra Bas study adopted the RPNs for child injury assessment and prioritization. RPNs used as risk factors specifically for child injuries as they integrate critical factors for risk assessment by defining scales of injury severity, detection, and probability clearly.[26] Lago et al. have used FMEA analysis to administer drugs in pediatric wards. Their proposed RPN also included RER related to failure mode probability to estimate occurrence and DBs in term of detected times to estimate the likelihood of detection.[7] Zammori and Gabbrielli proposed a new approach to split severity, occurrence, and detectability into sub-criteria and arranged them differently in a connective structure to calculate the RPN. Comparison considered between the importance of damages with respect to the goal, cause of failure, damage, and influence of each cause of failure on the others (domino effects).[27]

In this study, in addition to above items, repetition of doctor visits and increasing duration of operation were considered for severity. In terms of medico-legal consequences, Gal et al. compared people with disabilities to nondisabled persons in the field of affirming their voices and complaints and prosecuting them. In this study, complaints included court cases, written complaints to the hospital management, oral complaints, discontent, and lack of discontent.[28] We detailed all types of these complaints and additionally complaints to the Department of Health and the Medical Council. It is important to know that many of the failures occurred in our country would be resolved within the enterprises due to the medical community power and the lack of public awareness about the legal consequences and their rights. That is the reason why we considered more weight for time and cost. In addition, previous research has shown 41% of estimated failures directly or indirectly related to the time factor.[1] In this study, direct factors included delayed service delivery, lack of advanced planning for operation procedures and no waiting list, lack of intersectional coordination and lack of control mechanisms before starting surgical procedures. Indirect factors included situations which lead to time limitation, lack of sufficient time for compliance with protocols and standards of service delivery. Those situations often happen when there is a large number of patients, lack of adequate operative rooms, emergencies, staff shortage, and excessive workload. Time and cost factors were evaluated together as the more cost incurred by insurance companies, the more management time was required to solve related issues.

Hayes et al. multiplied the number of occurrences per year by the larger of the recovery or work around time. These were then given a scaling factor as more than a week, a full week, up to 2 days, up to a full day, up to an hour, and a brief interrupt.[28] Scales of OPT criterion in our study gives an example to allow the readers to extrapolate compatible to their own system.

Ding et al. estimates the risk/dose of carcinogens by considering both the tolerated and accepted risks with regard to the estimated risk/exposure.[24] So, risk exposure can be considered as a risk factor to calculate occurrence. As we studied a referral hospital, FGD members suggested that 1500 potentially exposed patient as the denominator of RER. That is because exposure to each failure varies for depending on the procedures. This makes no exposure possibilities for some patients and definitely exposure possibilities for some failures such as medical record and patient identification.

In field of detectability, we applied concepts in van Wagtendonk et al. study in which “nature, causes, and consequences of unintended events in surgical units” categorized into seven groups: Events types, consequences of events, reporters, root causes, phase of care, involvement role, degree of preventability [this category was applied in our study - Table 4].

FGD members add DBs as a sub-criterion to increase the validity of DS. However, if a system did not have an efficient reporting system (for AE, NM), it would be faced with many challenges. In addition to AE, reporting near misses is important to estimate DBs.

In Zammori and Gabbrielli study, weights for S, O factors are more than the weight of D in irreparable systems.[11] However, in Healthcare FMEA model equal weight is considered for each sub-criteria of patient outcome, visitor outcome, staff outcome, equipment or facility and fire.[29] Our study considered proportional weights for all sub-criteria according to their decisive role in health-care system.

We have distinguished between detectability and preventability to highlight DBs. Estimation of DB is dependent on having a Patient Safety Reporting System which details Near-misses (NMs) and AEs.[30] In the Healthcare FMEA model, detectability criterion has been eliminated because it was believed that the concept of detectability is hidden in occurrence and low detectability of many failures in healthcare systems.[29]

Conclusions

Risk assessment models which originated from industry must be adopted with regards to clinical contexts to make them more applicable to healthcare settings and results in more effective interventions. Application of any proposed model must scientifically be proved before its use in the health-care systems.

Limitation

In this study, the main concepts and sub-concepts of the tables can be used in other hospital wards or healthcare facilities, but scales might be changed proportional to professional team suggestion, the number of patients (for RER), the political, medicolegal factors affecting organizations, health-care incident reporting system (reporting NM or AE) and clinical field of study.

Financial support and sponsorship

Nil.

Conflicts of interest

There are no conflicts of interest.