Classifying Safety Events Related to Diagnostic Imaging From a Safety Reporting System Using a Human Factors Framework.

PURPOSE: The aim of this study was to measure diagnostic imaging safety events reported to an electronic safety reporting system and assess steps at which they occurred within the diagnostic imaging workflow and contributing sociotechnical factors.
METHODS: The authors evaluated all electronic safety reporting system reports related to diagnostic imaging during calendar year 2015 at an academic medical center with 50,000 admissions, 950,000 ambulatory visits, and 680,000 diagnostic imaging studies annually. Each report was assigned a harm score ranging from 0 to 4 by the reporter; scores of 2 (minor harm) to 4 (death) were classified as "potential harm." Two reviewers manually classified reports into steps involved in the diagnostic imaging chain and sociotechnical factors per the Systems Engineering Initiative for Patient Safety framework. The κ coefficient was used to measure interreviewer agreement on 10% of reports. The percentage of reports that could cause "potential harm" was compared for each step and sociotechnical factor using χ2 analysis.
RESULTS: Of 11,570 safety reports submitted in 2015, 854 (7%) were related to diagnostic imaging. Although the most common step was imaging procedure (54% of reports), potential harm occurred more in result communication (odds ratio, 2.36; P = .05). Person factors most commonly contributed to safety reports (71%). Potential harm occurred more in safety reports that were related to tasks compared with person factors (odds ratio, 5.03; P < .0001). The κ coefficient was 0.79.
CONCLUSIONS: Safety events were related to diagnostic imaging in 7% of reported events. Potential harm occurred primarily during imaging procedure and result communication. Safety events were attributed to multifactorial sociotechnical factors. Further work is necessary to decrease safety events related to diagnostic imaging.

Introduction

The use of diagnostic imaging has grown significantly in the last few decades, owing to technological advances in diagnostic imaging.(1, 2) A total of 124 million unique diagnostic imaging services (totaling $5.6 billion in Medicare payments) were performed for 34 million Medicare beneficiaries in 2014.(3) The growth in medical imaging has improved patient care, including more detailed views of the organ or body region of interest,(4, 5) but is also associated with some risks. However, patient safety risks, other than radiation exposure, have not been sufficiently researched.(6)

In many healthcare organizations, various patient safety events, including those that arise from diagnostic imaging, are being reported in electronic safety reporting systems. Such systems are commonly available resources for analyzing safety events, understanding errors and failures, and identifying how they can be prevented and corrected.(7, 8) Safety reports typically include the person(s) involved (and demographics), setting (e.g., outpatient), the event category, the location where the event occurred, a harm score, and a short narrative describing event details. Nursing staff are the most frequent reporters of safety events.(9, 10) However, any employee can file a safety report. Since the safety reporting system is voluntary and requires time to input the details, not all safety events at the study institution may be entered. Users can report on 15 self-identified categories of event types (e.g., Imaging, Coordination of Care, Diagnosis/Treatment, Facilities/Environment, Healthcare IT, Medication/IV Fluid, Identification/Documentation/Consent, and Surgery/Procedure). Our electronic safety reporting system is confidential, web-based and easily accessible at any institutional workstation. Reports are reviewed by the institution’s Department of Quality and Safety in a continuous effort to promote patient safety.

Human factors analysis(11, 12) provides a framework for understanding various structures, processes, and outcomes in patient safety that are contributors to potential harm. The Systems Engineering Initiative for Patient Safety (SEIPS) model focuses on healthcare as a socio-technical system in which a Person is but one component.(12, 13) However, the Person is central to the entire work system and is supported through the entire work system design. Socio-technical components within the work system include Person factors (e.g., providers and patients), Task factors (e.g., variety and complexity of tasks), Technology factors (e.g., health information technology [HIT](14)), Organization factors (e.g., institutional mandate for communicating critical results to ordering provider(15), care coordination), and Environmental factors (e.g., physical layout, noise). In the SEIPS model, work systems influence care processes, which contribute to patient, provider and organizational outcomes.(11, 13) The SEIPS model has been utilized to understand the management of electronic diagnostic test result notifications in the outpatient setting.(16)

We anticipated that an institutional electronic safety reporting system would provide sufficient information about diagnostic imaging cases to illuminate safety events related to diagnostic imaging. Assessing these safety events using the SEIPS model would enable a comprehensive understanding of factors that influence diagnostic imaging safety that would yield new insights in how they could be enhanced. The specific purpose of this study therefore was to: 1) measure the incidence of safety events related to diagnostic imaging from safety reports submitted to a voluntary electronic safety reporting system at a large academic medical institution, 2) assess the steps where they occurred within the diagnostic imaging workflow, and 3) assess contributing socio-technical factor(s) using the SEIPS model.

Methods

Study Setting

This Institutional Review Board-approved, Health Insurance Portability and Accountability Act-compliant retrospective cohort study was performed at a 793-bed urban, adult, academic, quaternary care hospital with approximately 50,000 admissions, 950,000 ambulatory visits, and 54,000 emergency department visits annually. The institution performs and interprets approximately 680,000 diagnostic imaging studies annually.

Data Source and Case Finding

Since 2004, the study institution has used the Electronic Safety Reporting System (ESRS), an electronic system for safety report submissions (RL Solutions, Cambridge, MA). Approximately 10,000 reports are submitted annually. A report is voluntarily submitted for safety events, defined as events that harm or could cause harm to a patient. The user specifies the harm score ranging from 0-4: 0=no harm-did not reach patient, 1=no harm-did reach patient, 2=temporary or minor harm, 3=permanent or major harm, and 4=death. In some cases, harm is “potential” and not “actual” harm to the patients. The scoring system can readily be mapped into the Safety Event Classification (SEC), an outcome-based classification system integrating taxonomies from the United Nations and the Joint Commission.(17) A near-miss safety event would have a harm score of 0; a precursor safety event would have a harm score of 1 or 2; a serious safety event would have a harm score of 3 or 4.

For this study, all safety events reported in the ESRS between the dates of January 1, 2015, and December 31, 2015, under the event category “Imaging” and all safety events reported in an imaging facility under any event category were included in the analysis. These safety reports were reviewed manually by two independent reviewers (RL and LC), with 10% of reports reviewed by both to assess inter-reviewer agreement. Some reports had been erroneously categorized into “Imaging” (e.g., “Surgery”-related events); these reports not related to diagnostic imaging were excluded during this manual review. In addition to the ESRS safety reports, the latest physician notes corresponding to the safety events from the patient electronic health records (EHR) were reviewed to assist in correctly categorizing the safety incident.

Event Classification

Each report was classified into the component of the diagnostic imaging workflow in which the safety event occurred, using a previously-described classification scheme.(18-20) It includes the following steps – Provider-Patient Interaction, Provider Discussion, Test Ordering, Test Scheduling, Test Protocoling, Imaging Procedure, Interpretation, Reporting, Report Communication. Classification was non-exclusive; if any diagnostic imaging safety event involved more than one step, it was classified into more than one step. Provider-Patient interaction events are those reported during a interactions between providers and patients while provider discussion events are events that occur during interactions between providers. Test ordering events included wrong orders (e.g., wrong side); test scheduling events included actual delays in care due to imaging scheduling. Test protocoling safety events were those where an erroneous protocol was selected during imaging. Imaging procedure errors included adverse reactions to intravenous contrast and any others safety events during an imaging procedure. Interpretation events are events relating to test interpretation, while reporting events are those that relate to documentation of reports (e.g. mislabeled, incorrect documentation). Report communication events included events relating to test result communication (e.g., imaging results that were not communicated to ordering physicians).

The reviewers also classified each safety event into which socio-technical factors, specifically SEIPS work system components, contributed to the event.(12) These include Person, Task, Technology, Organization and Environmental factors. A socio-technical work system component from SEIPS was relevant if any event could be attributed to it, as previously described.(12, 21, 22) This classification was also non-exclusive; if any diagnostic imaging safety event involved more than one socio-technical component, it was classified to more than one.

Data Analysis

The primary outcome measure was the incidence of safety reports related to diagnostic imaging, defined as the percentage of safety reports related to diagnostic imaging out of all safety reports submitted during the study period. We also measured the percentage of safety reports related to diagnostic imaging that could cause potential harm (i.e., with a harm score of 2-4) out of all reports related to diagnostic imaging. Secondary outcomes included the percentage of contributing factors to safety reports that could cause potential harm – including steps where an event occurred in the diagnostic imaging chain and contributing socio-technical work system components. We used chi-square analysis to assess differences in contributors to “potential harm” among safety reports related to diagnostic imaging, comparing the odds ratio (OR) of “potential harm” for events in various steps involved in the diagnostic imaging chain to the reference, “Imaging Procedure,” which was the diagnostic step involved in the greatest number of safety events. When expected cell counts fell below 1, Fisher’s exact test was used for statistical analysis. A p-value at or below 0.05 was considered statistically significant. Chi-square analysis also was used to compare differences in contributors to “potential harm” among contributing socio-technical work system components , comparing the OR of “potential harm” for each socio-technical work system component, compared to the reference, “Person factor,” which was the socio-technical work system component involved in the greatest number of safety events. We used kappa to measure inter-reviewer agreement on 10% of reports. Statistical analyses were performed using SAS 9.2 (SAS Institute).

Results

Study Cohort and Incidence of Diagnostic Imaging Safety Events

A total of 11,570 safety reports were submitted to the ESRS during the study period. Among these, 887 reports had been identified by the submitter as “Imaging” or was reported by the submitter to occur in an imaging facility. After manual review, 33 reports that were not related to diagnostic imaging were excluded, leaving 854 out of 11,570 (7%) safety reports from the ESRS pertaining to diagnostic imaging. 190 safety reports out of 854 (22%) related to diagnostic imaging were classified by submitters as those that could cause potential harm. Table 1 describes the patient demographics in the safety reports and the care settings (e.g. inpatient) where the events were reported. 55% of the patients were female and the mean age was 57.4.

Table 1:

Patient Safety Report Characteristics

Characteristic	Total (%)n=854
Patient Demographics
Age (mean ± standard deviation [years])	57.4 ± 16.7
Sex
Female	470 (55.0)
Male	384 (45.0)
Race
White	563 (65.9)
Black	55 (6.0)
Asian	23 (2.6)
Native American	2 (0.2)
Unknown/Other	177 (20.7)
Ethnicity
Latino	34 (3.9)
Care Setting
Inpatient	224(26.2)
Outpatient	615(72.0)
Not Specified	15(1.8)

Steps within the Diagnostic Imaging Workflow

The two independent reviewers manually analyzed 469 reports each. Inter-reviewer agreement for 84 cases revealed kappa=0.79 (95% Confidence Interval, 0.598-0.987). Out of the 854 total reports, 5 were classified under provider-patient interaction, 8 provider discussion, 387 test ordering, 24 test scheduling, 20 protocol selection, 464 pertaining to imaging procedure, 16 image interpretation, 15 reporting, and 22 report communication (Table 2). The most common step involved imaging procedure, and 138/464 (29%) of these safety events could potentially cause harm. Events classified by submitters as those that could cause potential harm occurred more in report communication (OR=2.36, p=0.05). These comprise 11 events related to report communication that were assigned a harm level of 2. These were attributed to imaging findings that were not communicated to the patient that they discovered on their own and test results that were not provided to care providers in a timely manner.

Table 2:

Classification of Safety Reports by Diagnostic Imaging Steps

Diagnostic Imaging Step	TotalCount	HarmLevel		PercentHarm	OddsRatio(OR)	95%ConfidenceInterval	p-value
		0-1	2-4
Provider-Patient Interaction	6	6	0	0%	0*	(0,0)	0.11
Provider Discussion	8	5	3	37.5%	1.42	(0.33, 6.01)	0.70
Test Ordering	384	376	8	2.1%	0.05	(0.02, 0.10)	<0.0001**
Test Scheduling	24	17	7	29.2%	0.97	(0.39, 2.40)	1.0
Test Protocoling	20	15	5	25.0%	0.79	(0.28, 2.21)	0.80
Imaging Procedure	464	326	138	29.7%	1.00		Reference
Interpretation	17	10	7	41.2%	1.65	(0.62, 4.43)	0.42
Reporting	15	15	0	0%	0*	(0,0)	0.01**
Report Communication	22	11	11	50.0%	2.36	(1.00, 5.58)	0.05**

When expected cell counts fell below 1, the Fisher Exact test was used for statistical analysis.

Statistically significantly different from the reference step (i.e., Imaging Procedure)

Several other safety events were reported in each step in the workflow: (1) Privacy curtain was not operational (provider-patient interaction); (2) Consultation with radiologist regarding renal function was not performed prior to administering contrast (provider discussion); (3) Test was ordered on the wrong side (test ordering); (4) Test was not scheduled for a patient who showed up for imaging (test scheduling); (5) Patient was given contrast for wrong imaging protocol (protocol selection); (6) Allergic contrast reaction (imaging procedure); (7) Initial test interpretation was discordant with final (image interpretation); (8) Inaccurate report documentation (reporting); and (9) Significant test result was not communicated with the patient (report communication).

SEIPS Work System Components

Classification with the SEIPS model demonstrated that the most common socio-technical work system component contributing to diagnostic imaging safety events was person factor (n=606), and 107/606 (18%) of these events could potentially result in harm (Table 3). 550 events were attributed to other non-person factors, of which 150 (27%) could potentially result in harm (chi-square analysis, p<0.0001). Specifically, task and internal environment were associated with more potential harm (OR=5.03, p<0.0001 and OR=6.22, p=0.02, respectively), compared to person factors. Task factor was associated with the most events with a harm level of 2-4 (Table 3). Examples of safety events related to diagnostic imaging are described in Table 4 with the corresponding work system component.

Table 3:

Classification of Diagnostic Safety Reports using the SEIPS Model

SEIPSComponent	TotalCount	Harm Level		PercentHarm	OddsRatio(OR)	95%ConfidenceInterval	p-value
		0-1	2-4
Person	606	499	107	17.7%	1		Reference
Organization	293	260	33	11.3%	0.59	(0.39, 0.90)	0.01**
Task	212	102	110	51.9%	5.03	(3.58, 7.07)	<0.0001**
Tools and Technologies	38	35	3	17.9%	0.40	(0.12, 1.32)	0.18
Internal Environment	7	3	4	57.1%	6.22	(1.37, 28.19)	0.02**

Statistically significantly different from the reference

Table 4:

Examples of Safety Events related to Diagnostic Imaging classified by SEIPS Work System Components

SEIPS Component	Elements	Description
Person	Skills	Order for imaging on the wrong side of the body
	Physical characteristics	Patient fainted during procedure
Organization	Coordination	Failure in informing patient of a scheduled imaging test
	Communication	Inadequate process for communicating abnormal finding to patient
Task	Variety of tasks	Delays in transport and scheduling transportation
	Job content	Hand-off is inadequate for staff who transported the patient
Tools and Technologies	Computerized Provider Order Entry	Imaging facility has limited access to patient orders
	Electronic Health Record	Results are not available in the Electronic Health Record
Internal Environment	Noise	Patients experienced too much noise from MR machine
	Layout	Power cord across the floor is a trip hazard

Discussion

In the study institution’s electronic safety reporting system, 7% of reported safety events involved diagnostic imaging. This provided an opportunity to report the incidence of safety events related to diagnostic imaging, identify where the events occurred in the workflow, and assign contributing socio-technical work system components.

The diagnostic imaging workflow includes several steps. Reported safety events occurred most commonly during the imaging procedure. As an example, this includes administration of intravenous contrast agents that produce allergic reactions. In severe cases, this includes patients who become unconscious during an imaging procedure. In spite of more safety events during the imaging procedure step, the risk of harm was greater in the report communication step. This step includes communication of critical imaging findings between providers and is key in informing providers, including orderers, primary care physicians, and specialists, regarding such findings in diagnostic imaging. This finding is not unexpected, given that most legal claims for outpatient care are related to missed or late diagnosis, including failure in communication leading to delayed diagnosis.(23, 24) Thus, the Joint Commission includes improved communication among its national patient safety goals, such as getting test results to the correct person quickly.(25)

Our institution has implemented several safety initiatives including a policy for critical imaging result notification, and an alert notification system, both associated with substantial improvements in critical imaging result notification to providers.(26, 27) However, this finding reiterates the importance of communication of follow-up testing, in order to ensure that results are monitored over time and that patients are involved in the diagnostic process. This study also emphasized that Tools and Technology, rather than improving communication, can lead to safety events when information is not readily available to appropriate providers. Ease of access and availability of information from Computerized Order Entry System and Electronic Health Records should be reviewed regularly to decrease safety events.

Safety events in diagnostic imaging were most commonly attributed to ‘Person’-related factors. A common example is data entry error for ordering a test on the wrong side of the body, for instance. These were frequently addressed at the imaging facility before the event reached the patient. However, most safety events that reporters classified at a harm level of 2-4 were most commonly attributed to ‘Task’-related factors. Complex tasks include patient transport problems leading to inadequate care, coordination of care and handoff (e.g., between radiology technician and respiratory therapist). Patient handoff is a known factor in patient safety.(28) Intra-hospital and other types of transport have also been known to affect patient safety,(29) although not in the diagnostic imaging domain.

These findings have important implications on effective interventions and workflow process redesigns based on human factors and high reliability design concepts and methods that minimize the possibility for and the consequence of human error. For example, identifying the above complex tasks in the diagnostic process as key factors in patient safety provides an opportunity to mitigate these factors. We expected that Information Technology was not sufficient to address failures in communication. To address the task of patient handoff between providers, further initiatives should ensure redundancy such that communication regarding testing, test interpretations, and follow-up recommendations are all completed. For each of these tasks, information technology for communication may be augmented with a care coordination process to catch any failures in communication. Overall the assessment of these events is a first and critical step in developing such approaches as well as evaluating the efficacy of any system level intervention.

This study had several limitations. We relied on safety events in the ESRS, which as mentioned previously, is not comprehensive in detecting all diagnostic imaging events. However, these data can be utilized in conjunction with other information sources to examine safety events in this domain. Under-reporting of safety events may be heterogeneous across various components of the imaging workflow, biasing our conclusions about relevant contribution of each component of imaging workflow to safety events. In addition, this study involves a single tertiary academic center. Thus, the incidence rates of reported safety events may not generalize to other institutions. However, we anticipate that safety events at other institutions will occur in various steps in the diagnostic process that involve similar socio-technical factors and thus be amenable to similar analyses.