Development of Hospital MCI and Disaster Preparedness Assessment Tool for Thailand.

INTRODUCTION: Hospital preparedness is a key component to mitigate the effects of mass casualty incidents (MCIs) and disasters. Improving hospital preparedness requires an assessment of the country's current health system capacity, readiness, and preparedness. Although a variety of assessment tools exist, none are entirely suitable for Thailand's healthcare system.
OBJECTIVE: To develop an assessment tool to evaluate hospital preparedness for MCI and disaster in Thailand.
METHODS: A cross-sectional study was conducted from 1 March 2015 to December 2016. The contents of the first drafted tool were prepared based on evidence from a systematic search of electronic databases published up to 31 December 2014. Key elements identification, extraction, and further organization were based on the World Health Organization health system framework. Validity was tested by experts and emergency management personnel in four domains using a 5-point scale evaluation form. The feasibility of using this assessment tool was carried out in 41 hospitals on a voluntary basis. The tool was considered valid if the item-objective congruence (IOC) index results were at least 0.6 and feasible for median values of at least 4.
RESULTS: Seventy-six full texts and guidelines out of 5869 titles and abstracts from a systematic search were enrolled in the study. A constructive literature review was performed to develop a hospital assessment tool. The IOC index results of the assessment tool components were 1.0, 0.9, 0.7, and 1.0 in framework appropriateness, relevance of items, clearness, and usefulness, respectively. The median (interquartile range) values of framework appropriateness, relevance of items, clearness, and usefulness were 4.0 (4.0‒5.0), 4.3 (4.3‒4.5), 4.0 (4.0‒4.0), and 5.0 (4.0‒5.0), respectively.
CONCLUSION: An assessment tool to evaluate hospital MCI and disaster preparedness based on the WHO health system framework was valid and feasible at the national level of Thailand.

Introduction

Disasters resulting from events due to natural and man-made hazards are occurring more frequently worldwide.1 The World Health Organization (WHO) implemented a risk-reduction and emergency preparedness plan for the health sector and community capacity development to assess emergency preparedness for mass casualty incidents (MCIs) or disasters in each country.2 MCIs and disasters are events that challenge local healthcare facilities and, potentially, entire health systems, as needs quickly exceed local capacities. This sudden imbalance occurs most typically within the first 48‒72 hours of an event before support becomes available from regional, national, and international organizations.3,4

Hospital preparedness is a part of community resilience and is a key component to mitigate the effects of MCIs and disasters in the care of injured patients.5,6 Recent reports indicate that many hospital emergency departments lack readiness for an MCI or a disaster, particularly those related to biochemical events.7–9 Just as an assessment of the Emergency Medical Services (EMS) system is performed using the EMS Incident Response and Readiness Assessment (EIRRA), hospitals must assess their capacity for management of an MCI or disaster.10 Although the National Institute for Emergency Medicine of Thailand implemented its 2013‒2016 Strategic Plan for Emergency and Disaster Preparedness,11 there is no recent evidence documenting the current level of hospital preparedness for MCIs and disasters in Thailand. This may be due, in part, to the lack of a standard tool that is appropriate to assess Thailand’s hospitals and the capacities of the healthcare systems.12

A variety of assessment tools were proposed by recognized organizations to assess hospital preparedness and readiness for either emergencies or disasters or both.13–19 The Hospital Safety Index (HSI) proposed by the World Health Organization (WHO) is one assessment tool widely used by many countries with favorable results. However, the HSI is not applicable for use in Thailand because many items are related to hazards not present in Thailand. Furthermore, the framework of the HSI differs substantially from the health system framework of Thailand’s Ministry of Public Health (MOPH). The HSI framework includes i) hazards, ii) structural and non-structural aspects, and iii) emergency and disaster management. On the other hand, Thailand’s health system framework includes i) governance and leadership, ii) health financing, iii) health workforce, iv) information system, v) medical products and technologies, vi) service delivery, and vii) participation.

Objective

This study aimed to develop a national hospital MCI and disaster preparedness assessment tool using the evidence from national and international literature and guidelines.

Methods

Study Design

A cross-sectional study was conducted from 1 March 2015 to December 2016. Ethics approval was obtained from the Institutional Ethics Committee Board of the Faculty of Medicine at Prince of Songkla University, Thailand (REC 57-314-20-1).

Development of the Hospital Assessment Tool

The contents in the assessment tool were initially developed by performing a systematic search from electronic databases: PubMed, Medline (Ovid), Cochrane Library (Wiley), Cinahl (Ebscohost), and Embase (Elsevier), and guidelines published by the WHO and other relevant organizations in disaster management up to 31 December 2014 in both English and Thai languages. Keywords used included assessment, evaluation, protocol, hospital preparedness, guideline, and full report. These terms were combined with disaster- or mass casualty-related keywords such as cyclone, tsunami, tornado, earthquake, flood, and terrorist.

Titles, abstracts, and full texts were evaluated and assessed independently by two of the authors (PW, RR) to identify articles related to medicine, public health, hospital, and disaster management. Any disagreements between the first two authors were assessed and decided by the third author (TK). The three authors (PW, RR, TK) independently reviewed, analyzed, and extracted key elements related to hospital preparedness in all phases of a disaster cycle and further organized into a drafted hospital assessment tool. Nominal group technique was conducted to draft the final assessment tool based on Thailand’s MOPH health system framework and the six building blocks developed by WHO (ie, governance and leadership, healthcare financing, health workforce, medical products and technologies, information system, service delivery, and plus participation). Finally, the first draft of the developed assessment tool consisted of four parts: 1) general descriptive information; 2) preparedness assessment checklist; 3) suggestions; and 4) hospital actual and surge capacity ().

Validity Test of the Assessment Tool

The validity of the tool was measured by three recognized experts from emergency medical service organizations, provincial health offices, and attending physicians of university hospitals using an evaluation form with four domains using a 5-point scale and an open-ended suggestion. The assessment tool was then modified before presenting it at the stakeholders meeting of 46 representatives of emergency management personnel from relevant organizations that included the Department of Disaster Prevention and Mitigation, World Health Organization (Thailand office), National Institute for Emergency Medicine, and Bureau of Public Health Emergency Response, Ministry of Public Health. The meeting took the form of a public hearing to obtain inputs from the stakeholders.

Feasibility Test of the Assessment Tool

The hospital assessment tool was distributed to the hospital director, emergency manager or chief of emergency unit of 41 hospitals. A total 33 (80%) hospitals volunteered to participate in the feasibility assessment.

Evaluation of the Assessment Tool for Validity and Feasibility

The four domains of the 5-point scale evaluation form with an open-ended suggestion were (1) framework appropriateness ranging from 1 (inappropriate) to 5 (most appropriate), (2) relevance of items ranging from 1 (irrelevant) to 5 (most relevant), (3) clearness ranging from 1 (unclear) to 5 (very clear), and (4) usefulness ranging from 1 (useless) to 5 (very useful).

Data Analysis

The data were entered into an Excel spreadsheet (Microsoft Corporation). Information from the 5-point scale evaluation form was categorized into 3 scores: 1 (relevant) if a scale of either 4 or 5, 0 (uncertain) if a scale of 3, and ‒1 (not relevant) if a scale of either 1 or 2. The item-objective congruence (IOC) index was calculated by the formula IOC = ΣR/N, where ΣR = sum of scores from the experts and N = number of experts. The tool was considered valid if the results were at least 0.6. The feasibility data were analyzed and reported as median and interquartile range (IQR). The tool was categorized as feasible for median values of at least 4.

Results

Seventy-six full texts and guidelines out of 5869 titles and abstracts from a systematic search were enrolled in the study for constructive review and development of a hospital assessment tool (Figure 1).

Figure 1

Literature searched, retrieved, and the selection process.

The final drafted hospital assessment tool was composed of 127 items of a preparedness checklist: Governance/Leadership (24 items), Financing (11 items), Health Workforce (14 items), Information Systems (17 items), Medical Products and Technologies (29 items), Service Delivery (25 items), and Participation (7 items) ().

The IOC index values assessed by the three experts for the appropriateness, relevance of items, clearness, and usefulness were 1.0, 0.9, 0.7, and 1.0, respectively (Table 1). The response rate from the feasibility study was 80%, and the study revealed all components in the hospital assessment were feasible with a scale of at least 4. The median (IQR) values of the framework appropriateness, relevance of items, clearness, and usefulness were 4.0 (4.0‒5.0), 4.3 (4.3‒4.5), 4.0 (4.0‒4.0), and 5.0 (4.0‒5.0), respectively.

Table 1

Findings from Validity and Feasibility Tests

Topic	Experts, IOC Index	Feasibility, Median (IQR)
Framework appropriateness	1.0	4.0 (4.0‒5.0)
Relevance of Items (overall)	0.95	4.3 (4.1‒4.5)
- Governance	1.0	4.0 (4.0‒5.0)
- Financing	1.0	4.0 (4.0‒5.0)
- Health workforce	1.0	4.0 (4.0‒5.0)
- Information system	0.67	4.0 (4.0‒4.5)
- Medical products and technologies	1.0	4.0 (4.0–5.0)
- Service delivery	1.0	4.0 (4.0‒5.0)
- Participation	1.0	4.0 (4.0‒4.0)
Clearness	0.67	4.0 (4.0‒4.0)
Usefulness	1.0	5.0 (4.0‒5.0)

Abbreviations: IOC, item-objective congruence; IQR, interquartile range.

Qualitative information from the experts and stakeholders in the validity and feasibility tests and details of the assessment tool modification are shown in Table 2. Furthermore, valuable comments and suggestions from the validity measurement endorsed the terminology of the framework and items in the preparedness checklist as part of the hospital assessment tool. The comments and suggestions from the feasibility study were used to modify and organize the hospital assessment tool. Ultimately, the hospital assessment tool was modified to become a manual for hospital preparedness for mass casualty incidents and disasters composed of concepts, purposes and framework, evaluators, terminology of components and key elements, and the hospital assessment tool.

Table 2

Qualitative Information from the Experts and Stakeholders in Validity and Feasibility Tests and Details of Assessment Tool Modification

Component	Qualitative Assessment			Tool Modification
	Experts	Stakeholders	Feasibility
General information		PAOs are part of health system		Added in general information
Governance	AAR is necessary in drill/exercise			AAR element was added (item 1.6.4)
			Coordination exercise is necessary	Coordination element was added (item 1.6.3)
			Incident Command System is missing	HICS element was added (item 1.7)
Financing	Emergency financial policy is one of the necessary elements			Engaged in budget management
Health workforce	Categorized into healthcare and supportive personnel		Specific and supportive teams are mandatory	Incorporated in item 3.1
Information system	Community and hospital capacity is mandatory	Include surge information	Include surge information	Information provided in hospital capacity
	Back up information system is necessary	Include back-up information system plan		The element was added (item 4.16)
	Experience in previous events is valuable	Experience in previous events is valuable		Integrated in disaster incident data (item 4.7)
	Medicine and logistic information are necessary	Lack of relevant organizations information		Added element in item 4.11
Medical products and technologies		Electric generator is necessary		Electric generator was added (item 5.10)
			Decontamination and radioactive room according to international standards	Elements were integrated in item 5.15
			Interpreters (vulnerability group and other languages)	Integrated in item 3.1.10, 5.19, 5.20
Service delivery			Include volunteer organizations	Element was added (item 6.11)
			Pediatric triage in disaster	As part of item 6.3
Participation			Coordination exercise is a necessary exercise	Integrated in item 7.1
			Stakeholder participation is necessary	Integrated in item 7.5, 7.6, 7.7
Others	Develop into scoring system	Develop into scoring system	Some hospitals have only a committee	Clarify terminology
		User manual for better understanding	Clarify emergency management team	Clarify terminology
		Specific tool for each hospital level	Develop hospital level-specific assessment tool	Further categorized tool for each hospital level
		Standardize terminology	Standard terminology	Constructed terminology

Abbreviations: PAO, Provincial Administrative Organization; AAR, After Action Review; HICS, Hospital Incident Command System.

Discussion

Effective management of the health consequences of emergencies and disasters requires a resilient health system. The Sixty-fourth World Health Assembly has urged countries to strengthen their disaster risk-management programs by integrating them into national health systems.20 Recognizing the need for strengthening health systems globally, and for all hospitals to create, revise, or update their respective disaster plans, the Pan American Health Organization and WHO developed the HSI, which is a standardized assessment tool based on an all-hazards approach and is deliberately generic in nature to allow modifications specific to the health system structure and resources of individual countries.21

Development of a standardized self-assessment tool based on the MOPH health system framework is needed to strengthen the hospitals in Thailand to prepare for either an MCI or disaster or both. Gaps and weaknesses need to be identified to guide hospitals to strengthen their capability and strategic planning as recommended in the systematic review of hospital disaster preparedness tools by Heidaranlu et al.22

The current hospital assessment tool was developed by a variety of methodologies with psychometric evaluations,22 which resulted in a valid and reliable tool. The tool was more scientific, evidence-based, valid, and feasible than the tools previously developed by a panel discussion of Thai experts.23 The systematic search provided fundamental evidence-based materials for the tool while the nominal group technique harmonized the tool’s concept and framework. The final drafted tool has suitable key elements in concordance with the country’s health system. The qualitative and quantitative assessments by the experts and stakeholders in emergency management have established the validity of the tool. The feasibility study produced a useful tool for the hospitals of Thailand. Most processes in the development and validity assessment of the hospital assessment tool were similar to the development of other tools, such as a tool to assess pediatric residents supervising inpatient rounds,24 an assessment tool of indicators for quality of care in severe preeclampsia/eclampsia and postpartum hemorrhage,25 and a forensic recording form for firearm injuries.26

The tool is concise and composed of the key elements in MCI and disaster management (eg, structure, functions, risk assessment, emergency and disaster management, actual and surge capacity, and participation).13–19,27,28 The tool is compatible with the HSI developed by WHO with arrangement of the items based on the MOPH health system framework and the six building blocks developed by WHO. The tool can be used for self-assessment of hospital preparedness and can help hospitals recognize their readiness, identify gaps for improvement as required, and allow for inputs for strategic planning. The six building blocks developed by WHO for a health system framework has been used in many countries and has proven to improve mass casualty and disaster management.29–32

Analyzing hospital preparedness and readiness for all-hazards mass casualty incidents and disasters by conducting a national survey would provide valuable information both to hospitals for better preparation and the MOPH of Thailand for policy guidance. The tool presented in the study can be applied in any nation with a similar health system framework to strengthen hospital preparedness and promote community resilience. Furthermore, for sustainable development and improvement, the tool needs to be incorporated into the hospital safety concepts outlined by Thailand’s MOPH and/or other organizations, especially the Hospital Accreditation Organization. A self-assessment scoring tool will be developed and a pilot feasibility study will be conducted. The result will compare the level of hospital preparedness for MCI and disaster among hospitals and provide guidance for a development of more comprehensive hospital assessment tool for MCI and disaster.

Limitations

This study has two limitations. First, publication bias was possible from the use of electronic databases that may not have contained unpublished but potentially useful tools or guidelines. Second, the response rate of only 80% from the feasibility test was used to finalize the assessment tool.

Conclusion

A hospital MCI and disaster preparedness assessment tool for Thailand was developed. The tool was valid and feasible to evaluate the current readiness of hospitals in Thailand. Use of the tool will provide essential information to strengthen the country’s health system and serve as a model in other low- to middle-income countries.