What a disaster?! Assessing utility of simulated disaster exercise and educational process for improving hospital preparedness.

INTRODUCTION: Recent events have brought disaster medicine into the public focus. Both the government and communities expect hospitals to be prepared to cope with all types of emergencies. Disaster simulations are the traditional method of testing hospital disaster plans, but a recent, comprehensive, literature review failed to find any substantial scientific data proving the benefit of these resource- and time-consuming exercises.
OBJECTIVES: The objective of this study was to test the hypothesis that an audiovisual presentation of the hospital disaster plans followed by a simulated disaster exercise and debriefing improved staff knowledge, confidence, and hospital preparedness for disasters.
METHODS: A survey of 50 members of the medical, nursing, and administrative staff were chosen from a pool of approximately 170 people likely to be in a position of responsibility in the event of a disaster. The pre-intervention survey tested factual knowledge as well as perceptions about individual and departmental preparedness. Post-intervention, the same 50 staff members were asked to repeat the survey, which included additional questions establishing their involvement in the exercise.
RESULTS: There were 50 pre-intervention tests and 42 post-intervention tests. The intervention resulted in a significant improvement in test pass rate: pre-intervention pass rate 9/50 (18%, 95% confidence interval ((CI) = 16.1-19.9%) versus post-intervention pass rate 21/42 (50%, 95% CI = 42.4-57.6%; chi-square test, p = 0.002). Emergency department (ED) staff had a stronger baseline knowledge than non-ED staff: ED pre-test mean value for scores = 12.1 versus non-ED scores of 6.2 (difference 5.9, 95% CI = 3.3-8.4); t-test, p <0.001. Those that attended > or = 1 component had a greater increase in mean scores: increase in mean attendees was 5.6, versus the scores of non-attendees of 2.7 (difference 2.9, 95% CI = 1.0-4.9); t-test, p = 0.004. There was no significant increase in the general perception of preparedness. However, the majority of those surveyed described the exercise of benefit to themselves (53.7%, 95% CI = 45.5-61.8%) and their department (63.2%, 95% CI = 53.5-72.8%).
CONCLUSIONS: The disaster exercise and educational process had the greatest benefit for individuals and departments involved directly. The intervention also prompted enterprise-wide review, and an upgrade of disaster plans at departmental levels. Pre-intervention knowledge scores were poor. Post-intervention knowledge base remained suboptimal, despite a statistically significant improvement. This study supports the widely held belief that disaster simulation is a worthwhile exercise, but more must be done. More time and resources must be dedicated to the increasingly important field of hospital disaster preparedness.