Training Situational Awareness for Patient Safety in a Room of Horrors: An Evaluation of a Low-Fidelity Simulation Method.

BACKGROUND: To protect patients from potential hazards of hospitalization, health care professionals need an adequate situational awareness. The Room of Horrors is a simulation-based method to train situational awareness that is little used in Switzerland.
OBJECTIVES: This study aimed to evaluate (1) the performance of health care staff in identifying patient safety hazards, (2) the participants' subjective experiences, and (3) the group interactions in Rooms of Horrors.
METHODS: The study was conducted in 13 Swiss hospitals that implemented a Room of Horrors. Health care professionals participated as individuals or in groups and were asked to identify as many errors as possible within a certain time and to complete an evaluation questionnaire. Observations of group interactions were carried out in 8 hospitals. t Tests and χ2 tests were used to examine differences in performance between participants solving the task alone versus in groups.
RESULTS: Data of 959 health care professionals were included in the analysis. Single participants identified on average 4.7 of the 10 errors and additional 10 errors and hazards that were not part of the official scenario. However, they also overestimated their performance, with 58% feeling the errors to be easy to find. Group observations indicated that participants rarely reflected on possible consequences of the hazards for the patient or their daily work. Participants feedback to the method was very positive.
CONCLUSIONS: Our findings suggest that the Room of Horrors is a popular and effective method to raise situational awareness for patient safety issues among health care staff. More attention should be given to debriefing after the experience and to benefits of interprofessional trainings.

Hospitalization puts patients at multiple iatrogenic risks, such as inappropriate hygiene precautions, unsafe environmental factors, or medication errors. During their stay, patients are thus potentially victims to processes of care that can cause harm and lead to adverse events. Although hospital staff knows about these potential patient safety hazards, in daily routine, they often remain unnoticed and uncorrected.[1] This can be ascribed to loss of situational awareness of health care staff. Situational awareness is based on the perception of environmental elements, “the comprehension of their meaning and the projection of their status in near future.”[2] Thus, both knowledge about patient safety hazards and situational awareness are crucial to protect patients from hospital-acquired harm. An innovative, low-fidelity method to increase situational awareness is the so-called Room of Horrors.[3] It is a patient safety simulation experience where participants enter a simulation room and identify as many errors and hazards as possible. A Room of Horrors offers a hands-on experience to train observational skills, critical thinking, and situational awareness concerning potential patient safety threats. In anglophone countries, the Room of Horrors is already in regular use, for example, for medical and nursing students or for entering interns. Different studies show that this approach is well appreciated among participants.[3-5] Although first single positive experiences exist in Switzerland,[6] the Room of Horrors has not been widely adopted to Swiss hospitals. More generally, evaluations of the concept are rare, usually being limited to single-center reports. In addition, most of the existing studies are focusing on participants’ performance and their feedback on the simulation experience. There are a few studies that take aspects of implementation in consideration, for instance, group trainings versus individual trainings or interprofessional groups versus monoprofessional groups. Little is known about how to best implement a Room of Horrors. The aim of this study was to evaluate (1) the performance of medical and health care staff in identifying errors and hazards across different institutions, (2) the participants’ subjective experiences, and (3) the group interactions in the Room of Horrors, that is, whether and how participants in groups work together to solve the task. We were also interested whether performance and subjective experiences differed between staff participating as individuals compared with those participating in groups.

METHODS

Design and Setting

This study was conducted within the context of the national patient safety week 2019, in which Swiss hospitals were encouraged to install a Room of Horrors (Fig. 1). The Swiss Patient Safety Foundation created a user manual for the hospitals to implement a Room of Horrors, including 6 case scenarios for different disciplines: internal medicine, orthopedics, cardiac surgery, geriatrics, pediatrics, and medication preparation. The manual is freely accessible in German,[7] French,[8] and Italian.[9] Each case scenario contained a patient case (a short medical record, medication, patient chart, etc.), a list of 10 errors and hazards, a list of materials to install the room, a participants’ instruction, a paper-based sheet to note the errors and hazards, a questionnaire to assess participants satisfaction, and a solution sheet (Fig. 2). All case scenarios were created in cooperation with experts from the hospitals, mainly nursing experts. The errors and hazards were chosen regarding their relevance for patient safety and their suitability for a Room of Horrors. They are partly based on findings from existing studies and reports,[4-6,10-12] partly on practical experience. In addition, we sought to find a similar distribution of errors and hazards within and between the patient case scenarios according to the following World Health Organization (WHO) incident types[13]: “clinical process/procedure,” “documentation,” “health care–associated infection,” “medication/IV fluids,” “nutrition,” and “patient accidents.”

FIGURE 1

Participants in the Room of Horrors at the University Hospital Basel, Switzerland. Photograph by Kenneth Nars. Reprinted with permission.

FIGURE 2

Scenario orthopedics solution sheet.

The medication preparation scenario differed from the other 5, as it does not describe a patient care situation but medication preparation in a simulated medication preparation room. Furthermore, it contained 20 errors and hazards, of which the participating hospitals should select 10 for implementation. These were mainly of the incident type “medication/IV fluids,” complemented by some of the type “infrastructure/building/fixtures.”

Procedures

Hospitals of various sizes and regions in Switzerland were asked whether they were interested in participating in the study, that is, in implementing one case scenario according to the manual. In addition to the 10 designated errors and hazards, they were free to implement 5 more errors and hazards with respect to their special institutional interest. All study hospitals were asked to invite particularly medical and nursing staff to participate in the Room of Horrors, but were free to extend the access to further staff such as therapists, dieticians, pharmaceutical personnel, and so on. It was not a mandatory requirement, but our recommendation was to participate in groups, ideally interprofessional ones, with a size of 3 to 5 persons.

Before the participants entered the Room of Horrors, they received a short instruction by the person in charge at the hospital, usually a clinical nursing expert. Participants in a simulated patient room were given 15 minutes, whereas participants in the medication preparation room were given 20 minutes to identify and document as many errors and hazards as possible. Documentation of identified hazards was done individually, even when participating as a group. Directly after leaving the room, participants were asked to complete a feedback survey. It was left up to the hospitals to decide how and when to communicate the solution of the implemented errors and hazards. Some decided to debrief immediately after participants had completed their search for errors; others preferred to wait until all participants had passed through the program.

Data Collection

Three different sources were used for data collection (Fig. 3): (1) participants’ documentation of identified errors and hazards, (2) feedback survey, and (3) direct observations.

FIGURE 3

Data collection.

The first data source was the paper sheets on which the participants documented all errors and hazards they had found during their visit in the Room of Horrors. In addition, participants were supposed to choose their profession from predefined options. The second data source was the questionnaire to assess participants’ subjective evaluation of the Room of Horrors as a training method. The questionnaire comprised 6 questions. Both, the sheets with the identified hazards as well as the questionnaire were completed anonymously by the participants and collected by the responsible person on site.

The third data source was observations of group processes carried out in those hospitals where participants solved the task as a group. Two observation sheets were developed, one focusing on the structure and the other on the content of group interactions. The first one comprised 7 questions. Questions 1 to 4 provided predefined options for categorizing the observations, for example, “the patient chart,” “the bed” or “the patient” (intensity of the searching activity), “group with a clear leader” (self-organization of the group), “(almost) all group members play an active part” (group activity), or “rather motivated” (mood barometer). The question on teaching activities asked for a count of the number of all observed teaching situations. Each question contained a free-text field for field notes. This observation sheet was filled in once per observed group. The second observation sheet consisted of a table with 5 questions and was filled in for every observed group interaction. Both observation sheets were tested using a Room of Horrors video. All observations were conducted by two experts, both having a clinical and research background. The observers received a briefing before the observations. All data were anonymized.

Data Analysis

For the error detection rates, we calculated the percent correct for each error or hazard in each scenario based on participants’ returned solution sheets. For example, if 20 individuals were exposed to a specific hazard of which 10 documented this hazard correctly, the percentage correct would be 50%. For the participants’ feedback survey, mean satisfaction scores are reported. t Tests and χ2 tests were used to examine differences in task performance and user experience between participants solving the task alone versus in groups.

RESULTS

A total of 13 hospitals participated in the study, 1 university hospital, 2 cantonal hospitals, and 10 regional hospitals. Ten hospitals (77%) were from the German and 3 (23%) from the French-speaking part of Switzerland. In 8 hospitals (62%), group interactions were being observed. Six hospitals chose the case scenario for orthopedics, 3 the scenario for internal medicine, 3 the medication preparation room, 1 the scenario for cardiac surgery, and 1 the scenario for pediatrics. One hospital installed 2 different rooms (internal medicine and orthopedics).

Of 992 participants, 959 returned a completed solution sheet. Most participants (64.5%) were registered nurses, 17.3% were health care assistants, 4.6% were medical doctors, 3.4% were physical therapists, 1.2% were pharmaceutical staff, and 9.0% had another professional background. In total, 11.2% of the participants were still in education. Most participants (n = 771) entered the room in groups of 2 to 7 individuals (208 groups) and 54 as individuals. The remaining participants (n = 134) could not be clearly assigned because of incomplete information. Most single participants completed the orthopedics scenario (41/54 [76%]).

Identification of Implemented Errors and Hazards

Variation Within Case Scenarios

Generally, the number of correctly detected errors and hazards varied widely within each scenario (Table 1). The greatest variability appeared in the pediatric case scenario, where no participant found the redness at the injection site of the peripheral venous catheter, whereas all identified the patient’s identification bracelet lying on the bedside table. In the cardiac surgery scenario, the underdosed torsemide and thus the risk of lung edema were least identified (2.6%), whereas 89.5% recognized the nurse call button out of reach. The latter was also the most frequently identified hazard in the orthopedics scenario (89.0%) and the internal medicine scenario (91.9%). In the internal medicine scenario, thus, the unnecessary urinary catheter was the least found error (7.3%), whereas in the orthopedic scenario, it was the missing deep venous thrombosis prophylaxis (17.6%). In the medication preparation scenario among all 3 installed rooms (Table 2), the incorrectly stored morphine was the most frequently identified error (91.53%; n = 354). However, only rarely noted were patient safety hazards concerning environmental factors that hindered a thorough medication preparation, for instance, an undersized work surface (room 1, 3.1%), poor lightening conditions (room 2, 0%), or the computer’s position located too far away from the preparation workspace (room 2, 0%).

TABLE 1

Correctly Identified Errors and Hazards of Hospitalization in Patient Rooms of Horrors

Errors and Hazards of Hospitalization	Percent of Correctly Identified Errors
	Internal Medicine	Orthopedics	Cardiac Surgery	Pediatrics
Incomplete PPE for isolation	86.9%	—	—	—
Urinary catheter without indication	7.3%	—	—	—
Intravenous administration of amoxicillin despite penicillin allergy	49.8%	—	—	—
Wheeled rollator left out of patient’s reach	35.9%	—	—	—
Nurse call button out of patient’s reach	91.9%	89.0%	89.5%
Missing of indicated blood glucose monitoring	31.3%	—	—	—
Double prescription of paracetamol	27.4%	—	—	—
Absence of indicated respiratory therapy	17.8%	—	—	—
Administration of wrong dose of amoxicillin	52.5%	—	—	—
Yoghurt in reach despite lactose intolerance	62.9%	—	—	—
Canes adjusted unequally	—	57.6%	—	—
Wrong patient name on blood specimen	—	73.5%	57.9%	—
Administration of the wrong intravenous drip	—	49.3%	—	—
Hazelnut yoghurt in reach despite nut allergy	—	72.8%	—	—
Lack of indicated DVT prophylaxis	—	17.6%	—	—
Absence of indicated pain assessment	—	22.1%	15.8%	33.3%
Wrong patient name in the patient record	—	27.6%	—	—
Incorrect surgical site marking (knee surgery)	—	65.9%	—	—
Empty bottle of hand sanitizer	—	49.0%	31.6%	—
Bloody wound dressing for thoracic trauma	—	—	63.2%	—
Underdosage of torsemide	—	—	2.6%	—
Missing cap on the 3-way tap of the central venous catheter	—	—	65.8%	—
Absence of indicated constipation prophylaxis	—	—	18.4%	—
Incorrect infusion rate	—	—	63.2%	—
Crispbread in reach despite gluten intolerance	—	—	57.9%	—
Aspiration risk due to inadequate patient’s positioning	—	—	—	38.9%
Normal consistency of food instead of pureed	—	—	—	88.9%
Incorrect dose calculation of ondansetron	—	—	—	55.6%
Filled syringe of ondansetron on bedside table	—	—	—	44.4%
Signs of phlebitis at the injection site of the PVC	—	—	—	0.0%
Patient’s identification wristband on the bedside table	—	—	—	100%
Absence of nose drops administration	—	—	—	5.6%
Overdosage of ibuprofen	—	—	—	77.78%
Risk of strangulation by intravenous tubes	—	—	—	72.2%
No. participants	259	290	38	18
Mean no. correctly identified errors and hazards per participant	4.64	5.24	4.66	5.17

DVT, deep venous thrombosis; PPE, personal protective equipment; PVC, peripheral venous catheter.

TABLE 2

Correctly Identified Errors and Hazards of Hospitalization in Medication Preparation Rooms of Horrors

Errors and Hazards of Hospitalization	Percent of Cases
	Room 1	Room 2	Room 3
Incorrect storage of morphine	92.6%	73.3%	91.4%
Expired drug in the pharmaceutical ward stock	58.9%	80.0%	—
Patient’s own, not clearly identifiable drug	62.4%	53.3%	—
Venlafaxine ER capsules prepared instead of tablets	12.0%	—	12.4%
Wrong patient name on intravenous infusion	77.5%	86.7%	72.8%
Incorrect infusion rate	66.7%	53.3%	—
Look-alike and sound-alike drugs stored next to each other	1.2%	46.6%	—
Double prescription of levocetirizine	8.9%	—	17.3%
Outdated guideline	22.1%	13.4%	—
Undersized work surface	3.1%	—	—
Empty hand sanitizer bottle	6.7%	—	—
Whole tablet prepared instead of a half tablet	—	53.3%	50.7%
Poor lighting conditions	—	0.0%	—
Co-amoxicillin prescribed as intravenous injection instead of short infusion	—	26.7%	95.1%
Computer located too far away from preparation workspace	—	0.0%	—
Loratadine missing	—	—	59.3%
Levothyroxin prepared without labeling next to the daily medication pill dispenser	—	—	77.8%
Danger of drug interaction between calcium und levothyroxin	—	—	11.1%
Underdosage of co-amoxicillin	—	—	34.6%
No. participants (n)	258	15	81
No. correctly identified errors and hazards per participant	4.03	4.93	5.22

Participants’ Performance Across Case Scenarios

On average (SD) and across all scenarios (Fig. 4), participants identified 4.7 (1.71) of the 10 implemented errors and hazards. More than half of the participants (69.0%) found 5 or less of the implemented errors and hazards, whereas less than one-third (31.0%) found more than half of the errors and hazards (Table 3). However, participants frequently identified additional problems that were not purposely implemented, for example, an inappropriate bed height, a missing resuscitation status in the patient documentation, hard-to-read handwritten notes, a warm yoghurt on the bedside table, a bed not labeled according to the latest guidelines, or a peripheral venous access on the hemiplegic side. On average, participants noted an additional 10 errors and hazards that were not part of the official scenario. Across all scenarios, participants in interprofessional groups (mean [SD], 5.1 [1.62]; n = 306) found significantly more errors and hazards than did those in the monoprofessional groups (mean [SD], 4.4 [1.63]; n = 465; t = −5.24, df = 769, P ≤ 0.001). Participants working in groups to solve the orthopedics scenario (mean [SD], 5.7 [1.76]; n = 146) found significantly higher numbers of errors and hazards of hospitalization compared with single participants (mean [SD], 4.7 [1.86]; n = 41; t = 3.18, df = 185, P = 0.0017). For the other scenarios, the fraction of single participants was too small for meaningful analyses.

FIGURE 4

Distribution of participants’ performance across all scenarios.

TABLE 3

Correctly Identified Errors and Hazards of Hospitalization Across All Scenarios

Sum of Correctly Identified Errors and Hazards of Hospitalization	Frequency	%	Cumulated
0	5	0.5	0.5
1	24	2.5	3.0
2	61	6.4	9.4
3	124	12.9	22.3
4	219	22.8	45.2
5	229	23.9	69.0
6	152	15.9	84.9
7	99	10.3	95.2
8	32	3.3	98.5
9	11	1.2	99.7
10	3	0.3	100.0
Total	959	100.0

Variation Between Types of Identified Errors and Hazards Across All Scenarios

Figure 5 shows that across all scenarios, errors and hazards of the WHO incident types[13] “documentation” and “clinical process/procedure” and “patient accidents” were found more frequently by individuals, whereas other incident types were identified more often by groups. Furthermore, there was a significant difference of detected errors and hazards between the WHO incident types. Most frequently identified were errors and hazards of the type “patient accidents” (69.3%), whereas errors and hazards of the type “infrastructure/building/fixtures” were least detected (3.8%). Less than half of the errors and hazards of the type “health care–associated infection” and “medication/IV fluids” were found across all scenarios.

FIGURE 5

Differences in group versus individual identification of errors and hazards according to the WHO incident types across all scenarios.

Participants’ Evaluation

The evaluation questionnaire was completed by 926 participants. Generally, their feedback concerning the Room of Horrors was very positive: 97.8% of the participants would recommend participation to their colleagues, 96.0% considered the implemented errors and hazards to be relevant for their daily work, and 94.9% reported that they had benefited from the group exchange and simulation training in general. The proportion of the participants who reported that the safety errors and hazards were easy to detect was 58.7%. Actual performance in error identification (number of hazards detected) and self-rated difficulty in detecting errors were not associated with each other (χ2 = 42.0, df = 30, P = 0.071). There were no significant differences in participants’ evaluation of their experience between individual and group participants. However, group participants were more likely to report that hazards were easy to detect compared with individual participants (59.7% versus 44.2%, χ2 = 4.8, df = 1, P = 0.028).

Group Interactions

Observations of group interactions were conducted in 8 hospitals and included a total of 83 groups. The mean (SD) group size was 3.8 (1.47) persons. As in the overall sample, most participants were nurses (47.1%), followed by health care assistants (23.9%) and doctors (11.0%). The 3 most popular ways of self-organization of the observed groups were the joint group search without clear leadership (31.4%), followed by searching in pairs plus individuals (19.3%) and the search of the entire group with a clear leader (16.9%). In a minority of 15.7%, the search was conducted as an individual task. A clear majority (95%) of the observed group members participated actively in the identification of errors and hazards; the mean (SD) rating of group interaction was 4.6 (1.34; with 1 indicating very low; and 6, very high). In the vast majority (98%), the group activity proceeded in a motivated, cheerful, and constructive manner. Furthermore, a mean (SD) of 4.4 (3.37) teaching situations per group could be observed in which one group member passed on his or her knowledge to the rest of the group. For instance, a health care assistant explained some features of the electronic patient documentation to a registered nurse, or a registered nurse informed the cleaning staff about different errors and hazards. Although group members often discussed reasons why they would consider something as a risk or hazard (70%), in only a quarter of the observed discussions, they talked about how to do it correctly, and in almost none of the observed discussions (1%–2%), they reflected on possible consequences for the patient or on similar situations in their daily work.

DISCUSSION

Our study is the first to report experiences and findings regarding the Room of Horrors method from multiple hospitals and a large group of participating staff. The results show that finding patient safety hazards is not an easy task for health care staff. Participants did not even correctly identify half of the intentionally implemented hazards. This detection rate is low but not astonishing, compared with 3 studies from the United States.[3,4,10] For instance, 49.8% participants in our study identified the administration of penicillin despite the patient’s penicillin allergy, whereas 53.6% in a study with interns[4] and 34.7% in a study with graduating nursing and medical students[10] found this error. We assume most participants knew that administering penicillin to a patient with penicillin allergy could be fatal. The findings indicate that there is a need of situational awareness training among health care staff. Recognizing patient safety hazards in the context of daily work requires certain skills that differ from theoretical knowledge. The Room of Horrors is a low-fidelity method that could help to identify blind spots and train such awareness skills. It is easily adaptable to different contexts and well received by health care staff. However, further research is needed to investigate whether training in a Room of Horrors has an effect on the frequency of certain incidents. In addition, more specialized Rooms of Horrors should be created for further disciplines or more in-depth education.

Interestingly, 58.7% of participants reported that the errors and hazards were easy to find, which is in contrast with their actual performance. The high self-estimation may be due to the many unintended errors and hazards that the participants found. These additional unintentional errors were very heterogeneous both within and between the scenarios and therefore seem to reflect the hospital-specific configuration of the room rather than having a direct association with the respective scenario. However, also Wiest et al[4] noted that interns who self-reported as being confident in their ability to detect hazards did not perform any better than did those with lower confidence. These results suggest a general poor awareness of potential safety hazards as well as an inadequate self-evaluation concerning the proper ability to recognize potential safety issues. The latter may be a consequence of not knowing what kind of errors and hazards they could have found but had not. In addition, not knowing which errors and hazards other participants had identified may induce participants to think it had been easy. Considering the findings of our observations that indicate participants’ low level of reflection of possible consequences of errors and hazards, we suggest an intensive debriefing directly after the visit in a Room of Horrors. It would help participants not only to put their own performance in relation to others but also to recognize own strengths and weaknesses, which may foster a better understanding for competencies of colleagues from different professions.

Some hazards were identified by most participants, whereas others were rarely found. This result is in line with international studies, although the errors and hazards mostly or least found differ between the different studies.[3-5,10] For instance, in our study, the unnecessary urinary catheter was identified by only 7.3% of participants, whereas the detection rate in other studies varied between 20.0%,[4] 20.1%,[10] and 36.3%.[3] Wiest et al[4] suggest that the results from the Room of Horrors simulation could guide health leaders and educators in identifying areas of focus toward providing high-value and safe care. Findings from a Room of Horrors could also be useful to start discussions about certain systemic aspects and thus create expedient improvement initiatives. For instance, the low detection rate of hazards of the type “infrastructure/building/fixtures” or the poor recognition of environmental factors that hinder a thorough medication preparation may be a result of “normalization” in health care staff toward unsafe systems and environments.

Our findings show that participants benefit from group interactions and find it easier to identify errors in groups. Furthermore, they found more errors and hazards if they were part of an interprofessional group. This supports the findings from a recent U.S.-American study with graduating medical and nursing students showing that interprofessional teams performed better than individuals in Rooms of Horrors.[10] The authors concluded that the Room of Horrors can help to learn the roles and responsibilities of different professions and to appreciate the shared goal. Our findings indicate that the Room of Horrors offers a practical and low-threshold possibility to teachings within a (interprofessional) group. We suggest integrating the Room of Horrors in interprofessional education and trainings programs to strengthen not only the awareness of hospitalization hazards but also the appreciation for each other’s roles and responsibities.[10]

There are several limitations to our study. Data are based on a convenience sample. The procedure in the hospitals differed: for some participants, the visit in the Room of Horrors was mandatory, whereas for others, it was voluntary. Potential confounders such as work experience, sex, and age were not considered in the analyses. There was a notable variability in the implementation of case scenarios between different hospitals.

It was not possible to compare the performance in the detection of errors and hazards between nurses and physicians directly, as the participation rate of physician was low and sometimes information on the profession was unclear or missing. The comparison between group and individual performance was limited to one case scenario. Furthermore, the observations were based on a subjective assessment of the observing person and were thus at risk of a personalized bias, which could have had an influence on the reliability of the observation data.

CONCLUSIONS

The Room of Horrors seems to be a popular and effective method to raise situational awareness for patient safety issues, although the self-overestimation of one’s own performance is in contrast with the actual performance of identifying hazards of hospitalization. The method can be easily adapted to different settings or groups and can be used for interprofessional trainings to improve not only situational awareness but also interprofessional teamwork and thus patient safety. To reinforce the learning effects, we suggest a high-quality debriefing directly after the simulation training. Further research is needed to find out how sustainable the effects of a Room of Horrors intervention are in terms of patient safety, for example, using a repeat set of “Rooms of Horrors” to see if the participants improved as a result of their experience.