Survey to identify depth of penetration of critical incident reporting systems in Austrian healthcare facilities.

Incident reporting systems or so-called critical incident reporting systems (CIRS) were first recommended for use in health care more than 15 years ago. The uses of these CIRS are highly variable among countries, ranging from being used to report critical incidents, falls, or sentinel events resulting in death. In Austria, CIRS have only been introduced to the health care sector relatively recently. The goal of this work, therefore, was to determine whether and specifically how CIRS are used in Austria. A working group from the Austrian Society for Quality and Safety in Healthcare (ASQS) developed a survey on the topic of CIRS to collect information on penetration of CIRS in general and on how CIRS reports are used to increase patient safety. Three hundred seventy-one health care professionals from 274 health care facilities were contacted via e-mail. Seventy-eight respondents (21.0%) completed the online survey, thereof 66 from hospitals and 12 from other facilities (outpatient clinics, nursing homes). In all, 64.1% of the respondents indicated that CIRS were used in the entire health care facility; 20.6% had not yet introduced CIRS and 15.4% used CIRS only in particular areas. Most often, critical incidents without any harm to patients were reported (76.9%); however, some health care facilities also use their CIRS to report patient falls (16.7%), needle stick injuries (17.9%), technical problems (51.3%), or critical incidents involving health care professionals. CIRS are not yet extensively or homogeneously used in Austria. Inconsistencies exist with respect to which events are reported as well as how they are followed up and reported to health care professionals. Further recommendations for general use are needed to support the dissemination in Austrian health care environments.

Introduction

Patient safety has increasingly become a priority issue, and national and international strategies have been considered to tackle the most prominent hazards.[1] Thereby, clinical risk management plays a dominant role in enabling the identification, analysis, and management of potential hazards.[1] To support clinical risk management and to identify potential hazards by health care professionals, so-called critical incident reporting systems (CIRS) were implemented. As early as 1999, in an “Institute of Medicine” report, a recommendation was made to focus more attention on CIRS.[2] CIRS were introduced as critical tools that could be used to identify potential hazards, referring to examples of their use in the aerospace industry.[3-5] So-called “near misses” (ie, “critical incidents” according to the definition provided by Medical Center for Quality in Medicine) that affect patient safety should be reported using CIRS.[6] A critical event is defined as an event that has the potential to do damage if no intervention occurs.[6] CIRS reports should be subsequently analyzed by experts, and their measures should be released in various forms to inform the employees of measures which were put in place due to a reported critical incident.[5-7]

In terms of their international use, CIRS are viewed as highly heterogeneous.[5,8] These differences are related to the fact that some countries legally require the use of CIRS, while others define the use of CIRS voluntarily.[9,10] Some organizations also define which types of critical incidents should be reported using a CIR-system[9,10]; in the latter case, the range of events reported extends from critical incidents without any harm to patients, and falls or those with sentinel events.[3,5,11,12]

After conducting semistructured interviews with experts in the field of patient safety, Mitchell et al reported that CIRS have not yet been used to their full potential.[2] According to this study, CIRS reports are (1) insufficiently analyzed, (2) physicians are insufficiently involved during the creation of reporting critical incidents, (3) the contents of CIRS reports are not sufficiently transparent, and (4) personnel at health care facilities receive insufficient support in using CIRS.[2,13,14] It is also evident that CIRS only captures a small fraction of occurring incidents in hospitals; reasons for nonreporting are a lack of feedback, lack of knowledge that a CIR-system is implemented, time pressure, and underestimation of critical incidents.[15] Therefore, it is utterly important that employees receive a proper feedback after reporting a critical incident about new measures as a cause of a critical incident; otherwise, employees do not perceive a benefit of reporting.

The accessibility of CIRS portals also varied. There are publicly accessible CIRS portals available via the Internet (Germany: CIRSMEDICAL.DE; Austria: CIRSMEDICAL.AT) as well as systems that are only accessible at certain health care facilities and, therefore, only accessible to employees with an Intranet account. In Switzerland, there is also an exchange portal for hospitals available, where critical incidents can be shared between participating health care facilities (CIRRNET). Even in a health care facility that has already introduced a CIRS, the CIR-system may only be accessible to employees in certain departments.

It is also questionable whether a high number of CIRS reports made in a health care facility is indicative of a more secure environment.[3,9,16] In any event, it has been shown that a higher number of CIRS reports is not associated with higher mortality rates.[9] In addition, a high number of CIRS reports on the same event indicates that critical incidents are being industriously reported, but that little is being done to create a safer environment or learn from errors.[3] Macrae further argued that, rather than emphasizing the quantity of CIRS reports, a focus should be placed on improving the quality of what is reported into a CIRS. The example of falls-reports clearly demonstrates this scenario in that it accounts for one-fifth of the 1.7 million CIRS notifications submitted to the National Health Service each year. According to an expert’s perspective, however, falls should only be reported in a separate, fall-related register.[3] However, a study also showed that a high CIRS reporting rate is associated with a positive environment of safety consciousness.[16] Despite the use of CIRS in the health care industry, it is still unclear how organizations can most effectively benefit from CIRS and how these CIRS reports can be used to improve patient safety.

In Austria, CIRS have only recently been introduced into the public health sector and are also used in various ways.[17] As described in a position paper prepared by the Federal Ministry, critical incidents and errors (sentinel events) should be reported into CIRS, in contrast to the classic recommendations.[5,7] The aim of this study, therefore, was to determine whether and how CIRS are used in Austria. Experts working in Austrian public health care facilities were requested to fill out a survey. Based on the information collected, a recommendation for the current and future use of CIRS is provided.

Methodology

According to the Ethical Committee of the Medical University of Graz, there is no legal requirement for an ethical vote as the survey did not include patients of the respective organization. A working group from the Austrian Society for Quality and Safety in Healthcare (ASQS) developed a survey addressing solely the topic of CIRS (Table 1) to assess the extent of CIRS usage as well as the specific use of CIRS reports.

Table 1.

CIRS Survey (Translation of Original Survey in the German Language).

1	Mark the type of facility in which you work: • Hospital • Nursing home • Outpatient clinic • Other (eg, Red Cross): (provide detailed information here)
2	Number of employees in your facility: • Up to 500 • Up to 1000 • Up to 3000 • More than 3000
3	Do you use CIRS in your facility? • Yes, in particular areas • Yes, throughout the entire facility • No, we plan to introduce CIRS within the next 2 years(go to end of the survey)  • No, there is no introduction planned within the next 2 years(go to end of the survey)
4	How are critical incidents reported in your CIR-system? • Only anonymously • Mostly anonymously • Mostly nonanonymously • Anonymously or nonanonymously
5	Who can report critical incidents in your CIR-system?(multiple answers possible)  • Employees • Patients and relatives • External service providers (eg, ambulance and cleaning service employees): (provide detailed information here)
6	What do you report into your CIR-system?(multiple answers possible)  • Events involving patients  ○ Critical incidents  ○ Cases of damage/harm • Patient falls  ○ Yes  ○ No • Events involving employees  ○ Critical incidents  ○ Cases of damage/harm • Needle stick injuries  ○ Yes  ○ No • Technical problems  ○ Yes  ○ No • Other  ○ (provide detailed information here)
7	How often do you derive measures and/or actions based on reported events?Never   Sometimes  Frequently   Always
8	How often is the efficacy of measures and/or actions controlled?Never  Sometimes  Frequently  Always
9	How do you check the efficacy of implemented measures?(If you answered “never” in Question 8, skip this question)  • Internal audits • Discussions • On-site inspections • Other (provide detailed information here)
10	How often are filed CIRS reports processed? • Daily • Weekly • Monthly • Per quarter
11	How are employees made aware of events reported in your CIR-system or implemented measures?(multiple answers possible)  • Via a portal accessible to all employees (ie, Intranet) • Employee newsletter • e-Learning • Via e-mail sent to respective area of the facility • Meetings/conferences • Individual CIRS/risk management reports • At this time, they are not made aware • Other (provide detailed information here)
12	How do you put together the team that processes the CIRS reports? • Employees working in QM or RM • Members of the quality assurance or quality management commission • Unique CIRS report processing team with(multiple answers possible)   ○ Representative physicians  ○ Representative nurses  ○ Representative Medical technical assistants  ○ Legal representative  ○ Representative member of quality or risk management  ○ Representative technician  ○ Representative member of administration  ○ Employee representative  ○ Representative of occupational medicine  ○ Representative of the management  ○ Invited experts  ○ Other (provide detailed information here)
13	Which information about CIRS is generally made available to employees in your facility?(multiple answers possible)  • General information about CIRS (Intranet) • CIRS handbook • e-Learning • Information events • Informative e-mailing • Newsletter • None • Other (provide detailed information here)
14	Which methods of risk analysis and/or assessment are used in your facility?(multiple answers possible)  • Top-down risk analysis • Failure mode and effects analysis (FMEA) • Root cause analysis • London Protocol • 3 F-method • Other (provide detailed information here)
15	Are CIRS results/statistics (in terms of relevant indicators) integrated into an information management or target control system?Yes, and if yes, which one? (provide detailed information here) No
16	Does your organization have a written agreement that addresses the freedom from sanction/confidentiality with regard to the use of the CIRS?Yes                 No

Note. CIRS = critical incident reporting systems.

All questions for the survey were developed by an ASQS task group and also incorporated questions from a survey (questions 5.1f, 5.1g, 5.1h, 5.1j, 5.1k, and 5.1l), which was recently performed to identify the penetration of clinical risk management in German hospitals.[18] Questions were transferred by a commissioned organization (MorethanChecks) into the form of a web-based survey instrument. Referring to the list of all health care facilities in Austria published by the Federal Ministry for Health and Women, 274 facilities were contacted by telephone to obtain the e-mail address of the person responsible for the CIRS or the quality or risk manager.[19]

ASQS sent the link to the web-based survey by e-mail; survey responses were collected anonymously over a 2-week period (June 28, 2016, to July 11, 2016), and 1 week after sending the link, a reminder e-mail was sent.

Results

In total, 371 health care professionals were contacted by e-mail, 78 of which completed and returned a survey (21.0% response rate). The number of contacted health care professionals exceeded the number of existing institutions in Austria because some institutions named several individuals as being responsible for their CIR-system (see Figure 1).

Figure 1.

STARD flow chart of the survey.

Most responses came from hospitals (n = 66; 84.6%). Responses from one outpatient clinic (1.2%), one nursing home (1.2%), and 10 other facilities (13.0%) were hereinafter assigned to the category “other facilities.” Most facilities indicated that they had 500 or more employees (hospital 500: n = 30; other 500: n = 10; hospital 1000: n = 11; hospital 3000: n = 7; other 3000: n = 1 [8.3%]; hospital >3000: n = 18; other >3000: n = 1).

A written agreement regarding the freedom of sanction and confidentiality with respect to the use of CIRS was available in 44 hospitals and 2 other facilities. In 56 hospitals and 6 other facilities, only employees could report into CIRS. Patients or relatives could also report into CIRS in 2 hospitals. This was impossible in all other facilities. External service providers could report a critical incident in 3 hospitals.

In 54 hospitals and 6 other facilities, only critical incidents without any harm to patients were provided, whereas in 15 hospitals and 3 other facilities, also critical incidents including harm to patients were reported. In 47 hospitals and 4 other facilities, also critical incidents without any harm to employees were reported. In 13 hospitals, also critical incidents including harm to the employee were reported. In 11 hospitals and 2 other facilities, patient falls were reported into CIRS. Needle stick injuries were reported into CIRS in 12 hospitals and 2 other facilities. In 35 hospitals and 5 other facilities, technical problems were reported into CIRS.

Results that could be extracted from the CIRS—in terms of identifying indicators relevant to management—were subsequently used in a management information or target control system in 34 hospitals and 3 other institutions. Additional survey results can be found in Table 2. The CIRS working and processing teams can be fundamentally assigned to 3 categories (see Table 3).

Table 2.

CIRS Survey Results (Part 1).

	Participating facility
	Hospital (n)	Other (n)	Total, n (%)
Do you use CIRS in your facility?
• Yes, throughout the entire facility	46	4	50 (64.1)
• Yes, in particular areas	10	2	12 (15.3)
• No, we plan to introduce CIRS within the next 2 years	7	1	8 (10.3)
• No, there is no introduction planned within the next 2 years	3	5	8 (10.3)
How are critical incidents reported in your CIR-system?
• Anonymously	29	2	31 (50)
• Mostly anonymously	12	4	16 (25.8)
• Mostly nonanonymously	14	0	14 (22.6)
• Anonymously or nonanonymously	1	0	1 (1.6)
How often do you derive measures and/or actions based on reported events?
• Always	34	2	36 (46.2)
• Sometimes	18	2	20 (25.6)
• Frequently	4	2	6 (7.7)
• Never	0	0	0 (0.0)
• Not answer	10	6	16 (20.5)
How often is the efficacy of measures and/or actions controlled?
• Always	27	3	30 (38.5)
• Sometimes	16	1	17 (21.8)
• Frequently	12	2	14 (17.9)
• Never	1	0	1 (1.3)
• Not answer	10	6	16 (20.5)
How do you check the efficacy of implemented measures?
• Discussions	15	4	19 (24.4)
• On-site inspections	18	0	18 (23.1)
• Internal audits	15	0	15 (19.2)
• Other	7	2	9 (11.5)
• Not answer	11	6	17 (21.8)
How often are filed CIRS reports processed?
• Weekly	20	1	21 (26.9)
• Daily	16	2	18 (23.1)
• Monthly	14	1	15 (19.2)
• Per quarter	6	2	8 (10.3)
• Not answer	10	6	16 (20.5)
How are employees made aware of events reported in your CIR-system or implemented measures?
• Via a portal accessible to all employees (ie, Intranet)	29	3	32 (20.5)
• Individual CIRS/risk management reports	24	4	28 (17.9)
• Via e-mail sent to respective area of the facility	27	0	27 (17.3)
• Meetings/conferences	23	3	26 (16.7)
• Employee newsletter	12	1	13 (8.3)
• Other	9	0	9 (5.7)
• e-Learning	0	0	0 (0.0)
• At this time, they are not made aware	5	0	5 (3.2)
• Not answer	10	6	16 (10.2)
Which information about CIRS is generally made available to employees in your facility?
• General information about CIRS (Intranet)	47	4	51 (27.4)
• Information events	31	3	34 (18.2)
• CIRS handbook	25	3	28 (15.1)
• Informative mailing	20	1	21 (11.2)
• Newsletter	17	3	20 (10.8)
• Other	11	1	12 (6.5)
• e-Learning	1	0	1 (0.5)
• None	3	0	3 (1.5)
• Not answer	10	6	16 (8.6)
Which methods of risk analysis and/or assessment are used in your facility?
• Failure mode and effects analysis (FMEA)	33	3	36 (27.1)
• London Protocol	23	4	27 (20.3)
• Other	17	1	18 (13.5)
• Top-down risk analysis	15	0	15 (11.3)
• Root cause analysis	11	3	14 (10.5)
• 3 F-method	7	0	7 (5.3)
• Not provided	10	6	16 (12.0)

Note. CIRS = critical incident reporting systems.

Table 3.

CIRS Survey Results (Part 2).

	Participating facilities
	Hospital (n)	Other (n)	Total, n (%)
How do you put together the team that processes the CIRS reports?
• Employees working in QM or RM	18	2	20 (8.7)
• Members of the quality assurance or quality management commission	7	0	7 (3.1)
• Unique CIRS report processing team with the following representatives	31	4	35 (15.3)
○ Representative physicians	30	4	34 (14.8)
○ Representative nurses	27	4	31 (13.5)
○ Representative member of quality or risk management	25	3	28 (12.2)
○ Representative Medical technical assistants	20	1	21 (9.2)
○ Representative member of administration	14	3	17 (7.4)
○ Invited experts	13	2	15 (6.6)
○ Representative technician	10	2	12 (5.2)
○ Legal representative	5	0	5 (2.2)
○ Other personnel	2	0	2 (0.9)
○ Employee representative	1	0	1 (0.4)
○ Representative of the management	1	0	1 (0.4)
○ Representative of occupational medicine	0	0	0 (0.0)

Note. CIRS = critical incident reporting systems.

Discussion

Although more than 15 years have elapsed since the publication of the report entitled “To Err Is Human” and the recommendation to introduce CIRS into public health services, only 64.1% of all surveyed respondents use a CIRS comprehensively in Austria. More than one-third of the surveyed health care facilities only use a CIRS in certain areas or not at all.

Provision of management support is critical to the introduction of CIRS. CIRS must be recognized as a strategic tool, which can be used to encourage learning processes throughout the organization in a long-term and effective manner. In addition, the existence of a written agreement regarding the freedom of sanctions and confidentiality with respect to use CIRS is essential. According to the survey results, not all health care facilities had such an agreement. Under no circumstances should the individual employee suffer a disadvantage by using a CIRS. Therefore, CIRS can only be effectively used if the management clearly demonstrates their willingness to address and respond to critical incidents without to sanction employees. This is the only way the individual employee can build up trust that he or she would report a critical incident.

In any case, yet another piece of the CIRS puzzle falls into place when one asks whether the anonymity of the reporting employee is preserved. The survey results showed that most CIRS reports are predominantly or purely anonymously, but that critical incidents can also be reported in ways that are not anonymous. The fact that some critical incidents are being reported nonanonymously is a sign that a certain safety culture exists, but nevertheless, special efforts should be made to preserve the anonymity of reporting individuals before any CIRS reports are edited.

CIRS were originally intended to be used by employees of health care facilities to report critical incidents. Upon close inspection, however, it becomes apparent that employees of many external service providers (eg, ambulance, cleaning staff) also work in these facilities. These external employees generally do not have access to a computer in the health care facility and, thus, cannot report any critical incident. They do, however, have contact with patients and should correspondingly be given the opportunity to report critical incidents. According to these survey results, this is currently practiced just in very few health care facilities. Furthermore, only employees and external employees working at a health care facility should have access to CIRS and it should not be used by patients or their relatives. Patients and their relatives are usually able to report complaints at the respective complaint management offices available in health care facilities, or they may visit the office of the ombudsman.

What should be reported into CIRS? According to the survey results, critical incidents that involved and, in some cases, events that resulted in harm to patients or employees were reported. Although CIRS were designed to facilitate the creation of reports about patients’ critical incidents, its use to report critical incidents involving employees also seems to be useful, because these reports are analyzed and processed in similar ways. However, it would be inadvisable to report cases of harm using CIRS, as these must be subjected to immediate legal investigations and can have far-reaching consequences for the patient, employee, and organization. To also disseminate results of legal investigations and their implemented measures, a mature clinical risk management system also facilitates a system change. As already described in the work of Macrae, falls, needle stick injuries, and technical problems should not be reported using CIRS; however, they should rather be reported in a separate register. Falls, needle stick injuries, and technical problems are events that frequently recur, floods CIRS, and increases the workload of CIRS processing teams.[4] In Austria, for example, nurses document each fall in a separate file in the respective hospital information system, needle stick injuries are documented and followed up by representatives of occupational medicine, and technical problems are part of an immediate reporting to the technical experts.

CIRS can only be used sustainably and as a supportive tool if CIRS reports are processed in a timely manner, and the efficacy of proposed measures is carefully controlled. In this way, managers and health care professionals will be encouraged to take CIRS reports and proposals of action based on them seriously. According to this survey, there are many kinds of CIRS reports available as well as there are many ways how measures may be controlled for their efficacy (see Table 2).

CIRS reports are processed by defined review teams (Table 3). These teams vary in terms of their size as well as their composition (ie, members of different professional groups represented). Representatives of each health care profession (eg, doctors, nurses, medical technical assistance) as well as the quality and risk management teams should be included in the working team. If possible, a representative from the legal department should also be involved to distinguish individual cases of critical incidents from those with harm. Other representatives from, for example, specialized medical fields may also be included, depending on the individual critical incident.

If complex cases are reported into CIRS, the analysis should be conducted using risk management tools, such as a process analysis using the failure mode and effects analysis (FMEA), London Protocol, or any other instrument that is generally used in the respective health care facility.

CIRS reports can play an important role in a learning organization by providing important information and serving as the basis of a standardized process taken to reduce critical events. For this reason, all employees must be made aware of completed CIRS reports via an Intranet portal, staff newspaper, e-mail, or by filing their individual CIRS reports. Annual CIRS case statistics should also be collected to provide the management with relevant information.

This study has several strengths and limitations. First of all, this is the first analysis to gain feedback if CIRS is generally used in Austria. It also provides a guideline (see Table 4) on how CIRS should be used to succeed in the Austrian health care sector. The poor response rate is the major limitation and did not improve despite sending a reminder to nonresponders. Invited health care professionals might have had certain concerns regarding anonymity, even though the survey process was outlined in detail. Furthermore, the medical society ASQS might not be known by all invited health care experts. Finally, we did not ask which CIRS-software is used or how frequent it is used. We believe that it is not the quantity of reports; it is all about the quality of reports.

Table 4.

Recommendations for General Practice of Using CIRS.

Chip 1—Responsibility of the management
1.1	Written agreement on confidentiality and freedom from sanctions
1.2	Comprehensive introduction of CIRS in all areas of an organization
1.3	In general, anonymous CIRS report filing
1.4	Regularly employees’ information and training on CIRS
1.5	Security culture promoted by management (learning from errors)
Chip 2—CIRS structure
2.1	All employees, including external service providers such as ambulance and cleaning staff, should be able to use CIRS
2.2	Reports on critical incidents involving patients and employees should be reported in CIRS
2.3	Cases of harm, falls, needle stick injuries (involving either employees or patients), or technical problems should not be reported in a CIR-system
2.4	The CIRS report processing team should be composed of (minimally) representatives of the health care professions (physician, nursing, medical technical assistant) and representatives from quality and/or risk management. If possible, a representative of the legal department should also be involved.
Chip 3—CIRS process
3.1	Analyze CIRS reports and use appropriate tools to process reports (FMEA, London Protocol, etc). Process CIRS reports in a timely manner
3.2	Create measures and/or action plans based on reported critical incidents
3.3	Provide CIRS reports to all employees
Chip 4—CIRS results
4.1	Check the efficacy of implemented measures (CIRS efficacy check)
4.2	Create a CIRS annual statistic report in terms of relevant indicators for the management

Note. CIRS = critical incident reporting systems; FMEA = failure mode and effects analysis.

Conclusions

In conclusion, it should be noted that our survey results indicated that CIRS are still not used extensively in Austria and, where used, its use is still highly heterogeneous. Inconsistencies exist with respect to how CIRS reports are filed and specifically which kinds of critical incidents are reported. It is, therefore, necessary to clarify what should be reported in a CIRS in Austria. Finally, CIRS should be made comprehensively available in each health care facility, so each individual area can “learn from mistakes.” In addition, external service providers should be given the opportunity to also report incidents. A CIR-system is an essential component of clinical risk management and is a simple tool that can be used to identify potential sources of critical incidents.[20] Finally, it is the aim of ASQS to disseminate the results of this survey to all Austrian health care facilities and to repeat the survey in due time.