Collaborative Case Review: A Systems-Based Approach to Patient Safety Event Investigation and Analysis.

OBJECTIVES: The aims of the study were to assess a system-based approach to event investigation and analysis-collaborative case reviews (CCRs)-and to measure impact of clinical specialty on strength of action items prescribed.
METHODS: A fully integrated CCR process, co-led by radiology and an institutional patient safety program, was implemented on November 1, 2017, at our large academic medical center for evaluating adverse events involving radiology. Quality and safety teams performed reviews for events identified with other departments who maintained their existing processes. This institutional review board-approved study describes the program, including percentage of CCR from an institutional Electronic Safety Reporting System, percentage of CCR per specialty, and action item completion rates and strength (e.g., stronger) based on a Veterans Administration-designed hierarchy. χ2 analysis assessed impact of clinical specialty on strength of action prescribed.
RESULTS: Seventy-three CCR in 2018 generated 260 action items from 10 specialties. Seventy percent (51/73) were adverse events identified through Electronic Safety Reporting System. The specialty most frequently associated with CCR was radiology (16/73, 22%). Most action items (204/260, 78%) were completed in 1 year; stronger action items were completed in 71 (27%) of 260. Radiology was responsible for 61 action items; 25 (41%) of 61 were strong versus all other specialties with strong action items in 46 (23%) of 199 (P < 0.01).
CONCLUSIONS: An integrated multispecialty CCR co-led by the radiology department and an institutional patient safety program was associated with a higher proportion of CCR, stronger action items, and higher action item completion rate versus other hospital departments. Active engagement in CCR can provide insights into addressing adverse events and promote patient safety.

Comprehensively addressing factors contributing to adverse events enables the development of interventions to eliminate or reduce the risk of recurrence and promotes patient safety and quality. Various system-based approaches have been used to understand these factors and identify actions to address them. Multiple approaches have been described, including root cause analyses (RCAs) and event investigation and analysis (EIA).[1,2] Focusing attention on contributing system factors, as opposed to solely investigating the performance of involved individuals, enables a fair and just culture to be established.[2,3]

The U.S. Department of Veterans Affairs and the Joint Commission introduced RCA to the medical community in the late 1990s.[2] Although not without its critics, RCA tools have been published and used regularly at the Veterans Affairs and elsewhere across the world since then.[4-11] The approach was adapted from aviation and focuses on 3 core questions to design safer care: “What happened?”, “Why did it happen?”, and “What action can we take to prevent it from happening again?”[12]

In addressing the first question, RCA includes assessing systems factors that are adapted from human factors engineering and incorporates questions about how systems are ideally supposed to work versus how they actually work day-to-day.[10,13,14] These factors include organizational rules and safeguards, environment, equipment, information technology, fatigue, scheduling, training, and communications.[14] Only after the problem is identified (“What happened?”) can the next questions be answered.

The second question (“Why did it happen?”) investigates root causes and contributing factors leading to an adverse event or near miss. Event investigation and analysis is a system-based approach initially described by the Agency for Healthcare Research and Quality, designed to enhance event reviews.[1] In this approach, the term “root causes” from RCA was replaced with “contributing factors” to more strongly manifest a system-based approach. Regardless of the terminology differences, both approaches identify contributing factors to find solutions—specifically ones that are effective and sustainable.[15] In both approaches, solutions are implemented as action items and are rated based on a hierarchy of the strength of actions.[12,16] These include stronger, intermediate, and weaker actions.

As part of an institutional system-based approach to patient safety, a multispecialty, integrated, collaborative EIA, co-led by the radiology department and the institutional quality and safety leadership and referred to as collaborative case reviews (CCRs), was implemented at our large academic medical center for evaluating significant adverse events. This study assessed a baseline for the number of action items performed by each clinical specialty and compared them with the specialty with the most action items, in response to each CCR. In addition, this study aimed to describe important elements of the CCR and measure the impact of clinical specialty on the strength of action items prescribed.

METHODS

Human Subjects and Setting

This institutional review board–approved, retrospective, observational study describes the institutional CCR process at an academic quaternary healthcare system. The hospital and ambulatory centers include a 793-bed hospital with approximately 50,000 admissions, and outpatient centers include approximately 950,000 ambulatory visits and 54,000 emergency department visits annually. Collaborative case review events are derived from the institutional safety reporting system, Electronic Safety Reporting System (ESRS; RL Solutions, Cambridge, Mass), as well as from other institutional reports of adverse events and near misses.

The ESRS has been in place since 2004. Approximately 10,000 reports are submitted annually.[13] Any employee can file a safety report in ESRS although nursing staff are the most frequent reporters, completing approximately 90% of reports filed. A report is voluntarily submitted for safety events, and these include actual adverse events where harm came to a patient and near misses. A near miss is defined as an unplanned event that did not result in injury, illness, or damage but had the potential to do so. Reports are investigated confidentially and addressed systematically by the institutional patient safety program, under the department of quality and safety.

Collaborative Case Review

A CCR is based on the EIA and RCA approaches for system-based analysis of factors underlying patient safety.[1,16] Each clinical department maintains processes for conducting CCRs.[1,2] The Patient Safety Program, responsible for conducting analysis of ESRS reports and learning from these events, works with individual departments regarding the events that they deem require a CCR (typically all sentinel events and adverse events with negative outcomes with potential hospital-wide implications). Collaborative case reviews are held on an as-needed basis when events are identified. Each CCR is led by a patient safety specialist who coordinates a multispecialty committee that is responsible for both describing the relevant factors leading to the event and identifying contributing factors to the event. These factors are assessed based on human factors analysis,[14] which entails investigating organizational processes and rules, environment, equipment, information technology, personnel (including cognitive factors, training, scheduling, communication), and individual tasks (e.g., complexity). If contributing factors are identified, then action items that may prevent similar events from occurring again are also established and individual departments or divisions are tasked with completing these action items in a timely manner.

For instance, standardizing the data format and requirements in radiology order requisitions was an action item assigned to radiology after a CCR identified that vague or unclear instructions on order requisitions was a contributing factor that led to an adverse event. Specific action items are proposed and decided upon at CCR meetings. These action items are grouped into primary analysis categorizations based on a strength hierarchy—stronger, intermediate, and weaker actions.[16] Stronger actions are those that remove dependence on humans to prevent an event.[16] These action items are typically more difficult to implement, but are felt to have a sustained and more successful impact, and can include things like engineering control and forcing functions (e.g., use equipment that can only be connected the correct way to a specific tubing), simplifying processes (e.g., removing unnecessary tasks in a multitask process), and standardizing equipment and processes (e.g., using standard medication pumps). Specifically, stronger actions force humans to make choices that minimize or eliminate the opportunity for wrong actions, often outside of the knowledge or control of the operator. These actions do not depend on the knowledge of an individual, require minimal supervision, and standardize processes through technology or design. In the example of standardizing radiology order requisitions, process standardization is expected to force the ordering clinician to provide the radiology department with accurate indications for scheduling the appropriate radiology examinations.

Intermediate actions reduce the reliance on humans but do not fully control for human error.[16] These include redundancy or back-up systems, increased staffing, software enhancements, checklists and cognitive aids, and enhanced documentation/communication. Weaker actions support processes but still rely primarily on humans.[16] These include processes such as double checks, warnings, trainings, and new procedures. A common example of a weaker action item is educating staff during an in-service training. Warning alerts are also weaker action items, especially if they can be overridden.

Department of Radiology CCR

Radiology department chair and vice chair for quality and safety promoted and encouraged submission of safety events by all staff so that through discovery and analysis, meaningful action items could be designed and implemented to improve the care of our patients. Beginning November 1, 2017, the entire CCR process in the department of radiology was fully integrated into the institutional patient safety program, co-led by the vice chair for quality and safety in radiology and the executive director of patient safety at the hospital. Both are present at each radiology CCR and are tasked with assembling a team of experts and staff to investigate each event. After each radiology-related CCR, the team assigns personnel who are responsible for all action items and due dates and monitors completion status and completion date. Ensuring that action items are completed and actively seeking stronger action items in response to safety events is explicitly encouraged by executive leadership in radiology. Alternative actions, when necessary, are identified and monitored as well.

In addition, the radiology CCR team works with the institutional patient safety program, which coordinates patient safety initiatives and operationalizes programs, leveraging resources from the entire institution. In addition to receiving requests for radiology CCR, the new CCR leadership team proactively looks at institutional safety events that involve radiology, conducting additional CCR when necessary on these safety events.

With enhanced access, events that are identified as requiring a CCR are readily provided to the radiology department’s CCR leadership. More CCRs are encouraged, which provides more opportunities for improvement. In addition, scheduling a CCR is streamlined by the departmental vice chair who co-leads the CCR. Once a CCR is initiated, all frontline providers and staff who were involved in the event are expected to prioritize meeting within a few days after the event to facilitate recall.

Collaborative Case Review Events and Study Outcomes

For this study, all CCRs performed in any department at the study institution between the dates of January 1, 2018, and December 31, 2018, were identified and included in the analysis. Each action item’s primary analysis categorization and strength level was recorded by clinical specialty. The primary outcomes were the percentage of CCR per clinical specialty and the incidence of CCR derived from the ESRS, defined as the percentage of CCRs that were initiated because of events reported in the ESRS. Secondary outcomes included completion rates of action items and strength of action items. Completion rates are measured at least 1 year after the analysis to enable sufficient time for clinical specialties to complete the actions. We report the strength of action items, which were measured as percentages of stronger, as opposed to intermediate and weaker actions.

Statistical Analysis

We used χ2 analysis to assess differences in completion rates of action items in different specialties. We further compared completion rates to the specialty with the most action items (i.e., comparator group). To detect a 50% effect size at 80% power and an α cutoff of 5%, 61 action items for the comparator group and at least 7 action items in another group were required. We also assessed the differences in percentage of stronger actions prescribed by different specialties. A P value at or less than 0.05 was considered statistically significant. Statistical analyses were performed using R statistical software (R Foundation for Statistical Computing, Vienna, Austria).

RESULTS

Collaborative Case Reviews Performed

We identified 73 CCRs performed at the study institution in 2018 from 10 clinical specialties (Table 1), including radiology, medicine, surgery, obstetrics and gynecology, emergency medicine, pediatric newborn medicine, nursing, anesthesia, pathology, and others (i.e., morgue, facilities, environment, engineering, and other staff). The most frequent clinical specialty associated with CCR was radiology (16/73, 22%). Fifty one (70%) of the 73 CCR performed were based on adverse events reported in the ESRS.

TABLE 1

Collaborative Case Reviews and ESRS Reports by Primary Clinical Specialty

Clinical Specialty	ESRS Reports	CCR Conducted, n (%)	Percent of CCR Based on ESRS Reports
Radiology	600	16 (21.9)	2.7
Medicine	5459	14 (19.2)	0.3
Surgery	3124	13 (17.8)	0.4
Obstetrics	789	8 (11.0)	1.0
Others	1015	6 (8.2)	0.6
Emergency medicine	496	5 (6.8)	1.0
Pediatric newborn medicine	408	4 (5.5)	1.0
Nursing	0	3 (4.1)	0
Anesthesia	95	2 (2.7)	2.1
Pathology	26	2 (2.7)	7.7
Total	12,012	73

Completion Rates of Action Items

A total of 260 action items were generated from the CCRs reviewed. The clinical specialty with the most action items was radiology (Table 2). Overall completion rate of action items was 0.78. Completion rates of action items varied by specialty, ranging 0.59 to 0.91 (P = 0.10). In comparison with radiology, medicine and emergency medicine had significantly lower completion rates. We were powered to detect a 50% effect size compared with radiology with 61 action items, except anesthesia with only 4 action items, powered at only 65%.

TABLE 2

Collaborative Case Reviews and Action Items by Clinical Specialty

Clinical Specialty	No. CCR Conducted	No. Action Items	Completed Action Items	Completion Rate	P
Radiology	16	61	54	0.89	REF
Medicine	14	54	38	0.70	0.02*
Surgery	13	47	36	0.77	0.10
Obstetrics	8	19	15	0.79	0.29
Others	6	12	11	0.91	0.75
Emergency medicine	5	22	13	0.59	<0.01*
Pediatric newborn medicine	4	23	21	0.91	0.71
Nursing	3	10	7	0.70	0.12
Anesthesia	2	4	3	0.75	0.43
Pathology	2	8	6	0.75	0.29

*Statistically significant.

REF, reference.

Strength of Action Items

Table 3 demonstrates the strength of action items implemented by each clinical specialty. Overall, 27% (71/260) of action items were stronger. The percentage of stronger action items, however, varied between specialties, ranging between 9% and 58% (P < 0.01). Obstetrics, radiology, and surgery implemented stronger action items compared with other specialties. Primary categorizations of stronger action items implemented in the CCR program include standardize process, simplify process, standardize communication, standardize equipment, engineering control, new device with usability testing, and tangible involvement by leadership.

TABLE 3

Strength of Action Items by Clinical Specialty

Clinical Specialty	No. Action Items	Stronger Action Items	Moderate-Weaker Action Items	Percentage of Stronger Action Items	P
Radiology	61	25	36	41%	REF
Medicine	54	10	44	19%	<0.01*
Surgery	47	12	35	26%	0.20
Obstetrics	19	11	8	58%	0.20
Others	12	2	10	17%	0.11
Emergency medicine	22	5	17	23%	0.13
Pediatric newborn medicine	23	2	21	9%	<0.01*
Nursing	10	1	9	10%	0.06
Anesthesia	4	1	3	25%	0.53
Pathology	8	2	6	25%	0.38
Total	260	71	189	27%

*Statistically significant.

REF, reference.

Table 4 shows examples of 2 event descriptions with action items that were implemented, including their level of strength. The first event was a delayed ultrasound that led to a delayed diagnosis of lymphoma. The second event was a delayed communication of critical findings, which subsequently led to a delay in management, necessitating resection of necrotic bowel.

TABLE 4

Examples of Events and Action Items by Level of Strength

Event Description	Action Item	Clinical Specialty	PAC	Strength
Patient was seen in clinic in September and had 1-mo delay in nephrology follow-up. Nephrology ordered renal ultrasound in the EHR in October, which was not performed until March the following year. Patient was diagnosed with lymphoma.	Development of a streamlined process across all clinics to ensure that patients are scheduled within e-referral indicated window (2 wk) and provide feedback to ordering provider if not completed.	Nephrology	Standardize process	Stronger
	Development of a streamlined process across all clinics for managing unscheduled orders.	Radiology	Standardize process	Stronger
	Increased awareness and timely response to patient messages regarding appointment scheduling.	Radiology	Education	Weaker
Preliminary read of CT scan was unremarkable; final read by attending noted bowel ischemia. Change in diagnosis and delay of report led to delay of patient going to the operating room by 12 h. Necrotic bowel was discovered intraoperatively.	Radiology implemented a stat study protocol that includes actionable reporting within 1 h of exam completion with closed-loop communication of critical findings. Process for radiology subspecialty attending to referring attending communication established	Radiology/surgery	New procedure	Weaker
	The radiology department will monitor use of stat orders for appropriateness and provide feedback to ordering providers and services if there is overuse of stat orders.	Radiology/surgery	New procedure	Weaker
	The radiology department requested removal of structured information from radiology ordering system and requires providers to use free text to describe indications for study, as a substantial portion of structured indications for imaging exams selected by ordering providers contradicted documented ordering provider clinical notes in the EHR.	Radiology	Standardize process	Stronger

CT, computed tomography; EHR, electronic health record; PAC, primary analysis categorizations.

DISCUSSION

At our institution, 73 CCR reviews were completed in a year, accounting for a total of 260 action items. Seventy-eight percent of these were completed in 1 year, and 27% were categorized as stronger action items based on a published strength hierarchy.[16] In its inaugural year, our radiology CCR program conducted 21.9% of the 73 reviews, with radiology having the most CCR of all clinical specialties. Obstetrics, radiology, and surgery implemented stronger action items compared with other specialties.

The CCR program was essential in addressing major safety events that were reported in a safety reporting system. The Joint Commission’s sentinel event policy requires healthcare organizations to perform RCA for sentinel events—safety events that lead to death, permanent harm, or severe temporary harm.[17,18] More importantly, it expects submission of action items in response to an in-depth investigation of these events. Our CCR program monitors action items that are prescribed for safety events and monitors these actions until completion. Our 78% completion rate is comparable with completion rates reported in other published studies.[12,19,20] Completion rates did not vary significantly by clinical specialty.

Action items are not equally effective in addressing safety events and mitigating risks.[12] In many studies, policy changes, education, and training accounted for a vast number of action items.[6,19,20] These interventions, however, are weaker actions with a lower chance of reducing risk eliminating repeat injuries. Thus, the goal is to recommend stronger action items whenever possible. In this study, we noted that pediatric newborn had one of the highest action item completion rates at 91% among specialties. However, only 2 (9%) of the 23 action items implemented were stronger action items. Forty-one percent of all action items implemented in radiology and 58% of action items implemented in obstetrics were stronger action items. In obstetrics, there was active leadership support for CCR from an engaged physician quality lead. The percentage of stronger action items is higher than that typically reported and among the highest among our clinical departments but reveals that there may be an opportunity to do even better institution-wide.

An integrated CCR co-led by the radiology department and the institutional patient safety program was responsible for a greater proportion of CCRs performed, compared with other specialties, and had action items that were stronger and do not rely primarily on humans to prevent or replicate an adverse event. The majority of our stronger action items belong to the standardize process primary analysis categorization. We addressed safety events such as delays in diagnostic testing by creating standardized processes that mitigate risk and prevent these events from reoccurring. Process standardization is pursued by ensuring that leadership at all levels of the institution supports the process and is an active participant in standardization together with a project core team from appropriate disciplines.

As an example of a safety event, we described a delayed ultrasound that led to a delayed diagnosis of lymphoma in our institution. The ordered exam was unscheduled for months, and we have demonstrated that 7% of our diagnostic imaging orders remain unscheduled.[21] Most orders remained clinically necessary. This could be a large contributor to delay in diagnosis, an important patient safety concern.[22,23] Thus, stronger actions to address unscheduled orders should be a priority. Our integrated CCR used a standardize process primary analysis categorization by initiating a streamlined process across all clinics for managing unscheduled orders. We have, since then, initiated and described a system for coordinating orders for radiology examinations, which resulted in 49% reduction in unscheduled orders at our institution.[24] Our system ensures that examinations ordered by providers are executed in a timely fashion to reduce diagnostic delays for clinically necessary orders and that clinically unnecessary orders, such as duplicate orders, are removed expeditiously to minimize medical errors. Further studies to evaluate the system will strengthen its impact on reducing delays in diagnosis and enhancing patient safety.

Limitations of this study include not assessing how well each CCR was conducted and how comprehensive the human factors analysis was. We also did not assess whether the action items to address these factors were appropriate. Finally, we did not measure direct impact of the action items on patient safety outcomes or risk of harm in the future. However, we reported process measures, which include percentage of action items categorized as stronger actions, one key measure of effectiveness and sustainability of RCA.

CONCLUSIONS

An integrated CCR co-led by the radiology department and institutional quality and safety leadership was associated with higher percentage of CCRs performed, stronger action items being recommended, and higher action item completion rates compared with other hospital departments. Active engagement of radiology departmental leadership in strong collaboration with institutional safety leadership for timely performance of CCRs highlighted the importance of this process to all departmental staff. Promoting CCRs and monitoring completion rates and strength and categories of action items via CCRs can provide insights into identifying factors that can promote patient safety.