Preparedness of community-based organisations in biohazard: reliability and validity of an assessment tool.

The purpose of this study was to develop a tool for community-based health organisations (CBHOs) to evaluate the preparedness in biohazards concerning epidemics or bioterrorism. We searched concepts on partnerships of CBHOs with health systems in guidelines of the Centers for Disease Control and Prevention and literature. Then, we validated the researcher-made tool by face validity, content validity, exploratory factor analysis (EFA), confirmatory factor analysis (CFA) and criterion validity. Data were collected by sending the tool to 620 CBHOs serving under supervision of Iran's ministry of health. Opinions of health professionals and stakeholders in CBHOs were used to assess face and content validity. Factor loads in EFA were based on three-factor structure that verified by CFA. We used SPSS V.18 and Mplus7 software for statistical analysis. About 105 health-based CBHOs participated. After conducting face validity and calculating content validity ratio and content validity index, we reached 54 items in the field of planning, training and infrastructure. We conducted construct validity using 105 CBHOs. Three items exchanged between the fields according to factor loads in EFA, and CFA verified the model fit as Comparative Fit Index, Tucker-Lewis index and root mean square error of approximation were 0.921, 0918 and 0.052, respectively. The Cronbach's of the whole tool was 0.944. Spearman correlation coefficient confirmed criterion validity as coefficient was 0.736. Planning, training and infrastructure fields are the most important aspects of preparedness in health-based CBHOs. Applying the new assessment tool in future studies will show the weaknesses and capabilities of health-based CBHOs in biohazard and clear necessary intervention actions for health authorities. © Author(s) (or their employer(s)) 2019. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

Introduction

Community-based organisations (CBOs) are indispensable partners in health systems.1 In disasters and management of epidemics, some governments cannot provide an adequate response with limited resources.2 These organisations have the necessary knowledge about community culture, structure and resources, and facilitate access to deprived and marginalised communities in times of need.3 4 Besides, Federal Emergency Management Agency and Centers for Disease Control and Prevention (CDC) have encouraged coordination with volunteer organisations whose goal is to support and coordinate with government agencies in disasters.1 5

Contrary to these requirements, there is limited evidence regarding the readiness of these organisations for bioterrorism or epidemic/pandemic.6 For instance, in order to prevent the spread of Zika virus, organisations are expected to have very high communication skills to warn high-risk groups. In a study by Zhi et al; they emphasised the need to educate the staff of faith-based organisations especially in exercises and drills.7 Eventually, Clawson et al examined the community health centres’ preparedness in the case of bioterrorism. They demonstrated that less than half of the centres possessed bioterrorism preparedness in their plan and only one-third among them included bioterrorism preparedness in their written policies.8

Frequent religious mass gatherings and food serving in customs gathering exposure Iran in the various potentiality of outbreaks and bioterrorism.9–11 Besides, limited access to resources in epidemic seasons in developing countries like Iran would worsen the situation.12 Although Iran has an extensive community-based primary healthcare (PHC) network that raise community awareness of local risk profiles and aid community, a population-based study revealed that overall community awareness and preparedness for even routine disasters is low.13–15 Iran’s PHC network has taken some advocacy and training programmes focusing on community partnership.13 14 CBOs can equip public health officials with information about vulnerable groups and how to meet their particular needs.16 However, Iran still lacks an assessment tool evaluating the preparedness of CBOs in biohazards.

According to the above evidence, the necessity of formulating specific criteria in the context of preparedness for CBOs is obvious. Therefore, this study aimed to develop a comprehensive tool to evaluate the level of CBHOs' preparedness in times of epidemics or bioterrorism thereby identifying the prerequisites of CBHOs’ participation for government decision makers.

Method

Participants and sampling method

The study population was 620 CBHOs in the country. Based on Cochrane’s formula,17 approximately 100 samples should be included in the study. Receiving a compiled list of CBOs from vice-chancellery for social affairs, the tool was disseminated in CBHOs’ social networks. Additionally, we sent the participation appeal through contact channels five times for each CBHOs. These are organisations that, according to experts from the Centers for Disease Control at deputy of health, possess the eligibility and capabilities necessary to work with the health system as an assisting or cooperating agencies during biohazards. Inclusion criteria were: CBHOs that served more than 50 clients and provide services to clients at the time of the study with a registered office to carry out their duties. The identity of the person filling the questionnaire and their organisation remained disclosed in the data collection forms; instead, a code was assigned at the time of data entry.

Instrument

Face validity, content validity, exploratory factor analysis (EFA), confirmatory factor analysis (CFA) and criterion validity were used to validate this researcher-made tool. Assessing the face validity, health professionals in disasters/emergencies and some stakeholders from CBHOs read questions of the instrument to examine the level of difficulty, the degree of mismatches and to check the ambiguity of phrases and meanings of words,18 thereby making appropriate changes in Persian based on their comments. The qualitative content validity determined by the grammar, the proper use of words, the importance of questions, the ordering of questions and the time required to complete the toolkit were all taken into consideration.19 20 For quantitative content validity, content validity ratio (CVR) was used to ensure that the most important content was chosen (necessity of question), and the content validity index (CVI) was used to ensure using the best way to measure content.21 22 CVR is used to ensure that the most relevant and correct content is selected.21 23 In quantitative content validity, after collecting expert opinions, if the CVR based on Lawshe’s table was more than 62%24 and the CVI based on the Davis study was over 80%,22 the necessity, relevance, transparency and simplicity of the questions were acceptable.

We used EFA to determine the factors loads based on three factors extracted as the assumption of this study. CFA, also, was used to verify the factor structure and the hypothesis that a relationship between observed variables and their underlying latent constructs exists.25

In order to assess the concurrent criterion validity, the ‘organisational preparedness checklist for a major earthquake or other large-scale disaster events’ questionnaire developed by Austin et al was applied.26 The questionnaire consists of 30 phrases in four periods that include ‘Last year’, ‘Between 1 year to 3 years’, ‘Over the past 3 years’ and ‘Never’.

Data collection

The researcher-made tool was designed to determine the preparedness level of CBHOs in biohazards. After studying the guidelines of Office of the Assistant Secretary for Preparedness and Response (ASPR)27–29 along with other information sources, the research team eliminated duplicates/repetition of initial items pool.

The researchers extracted all concepts promoting the partnership and cooperation of CBOs with the health system under the first set of capabilities named ‘community preparedness’ in the Public Health Preparedness capabilities guideline. Then, fundamental concepts of the tools that are available from the literature were integrated into the primary tool to evaluate CBHOs’ preparedness.

Statistical analysis

We evaluated the construct validity with EFA in SPSS V.18 software and CFA in Mplus7 software (Muthén and Muthén, Los Angeles, California, USA). Fitting indexes included Comparative Fit Index (CFI), Tucker-Lewis index comparative fit index (TLI), root mean square error of approximation (RMSEA) and weighted root mean square residual. Cronbach’s α was used to measure the internal consistency.

Results

Model assumption

At first, we obtained 115 items. There were 56 items in the planning field, 21 items in the field of education and 38 items under infrastructure. For each item, a range of four options was considered, including ‘not done’, ‘due to review’, ‘planned but not implemented’ and ‘completely implemented’.

Face and content validity

In face validity, four items in the planning field, one item in the field of training and two items in the infrastructure field were omitted. After reviewing the questions, 10 healthcare professionals who specialised in the field of disasters and emergencies inspected the qualitative content validity. Following this, based on the comments, certain corrections were made regarding the wording and language used in the questionnaire. In quantitative content validity, based on the CVR and CVI indicators, 21 items in the planning field, 10 items in the field of training and 24 items in the infrastructure field were deleted. Finally, 54 items remained for entry into the construct validity stage.

EFA and CFA

About 105 CBHOs volunteered to participate in the study. The Kaiser-Meyer-Olkin test for sampling adequacy was 0.773. Next, to determine if the correlation matrix had a significant difference with zero and factor analysis was justifiable or not, the Bartlett test of sphericity was performed, which turned out to be 4293 χ2 and p<0.001. These values ​​indicated that the factor analysis is justifiable based on the correlation matrix, and the items can be used for factor analysis. In this study, limiting the extracting factors by referring to the ‘community preparedness’ principle of the CDC guideline in the field of public health emergencies27 and applying varimax rotation, EFA with three factors were performed.

We considered an inflection point of 0.3 as the minimum factor load needed to maintain the items. Thus, 54 items without deletion were entered into the CFA stage. Two questions of the planning field and one question of the infrastructure field were exchanged based on the rotation matrix and the loadings factor, which were conceptually meaningful. The questions 30 in the table 1 and 12 and 13 in table 2 were those exchanged questions. All 11 items in the training field remained in the factor structure according to factor loads (table 3). The three-factor model confirmed by deleting item 13 in the substructure field based on CFA findings (table 2). Table 4 shows CFA fitting indexes.

Table 1

Factor loadings of planning field

Items	Factor loading
1. Do you have a list of necessary telephone numbers to contact the responsible organisations during emergencies?	0.618
2. In biohazards, is there a plan to send warning messages to personnel?	0.493
3. Is there a specific committee or chain of command for emergency preparedness?	0.474
4. Is it possible to contact the person in charge in 24 hours?	0.432
5. Did the role and responsibility of your organisation define to other responsible organisations?	0.541
6. Is there a specific method for informing your organisation in an emergency?	0.631
7. Has a written response and preparedness plan been developed for personnel?	0.581
8. Is there a plan to follow the immunisation of personnel in the early hours of a biological incident?	0.535
9. Is there a plan to approve the authorisation of clinical volunteer staff during a biological incident?	0.612
10. In biological incidents, is there a plan to cover the clients' needs?	0.669
11. In biological incidents, is there a plan to provide mental support for the covered population?	0.544
12. Do you have any plans to support the mental and moral well-being of children during an emergency?	0.698
13. Is there a plan to care for children whose guardians have died or confined for a long time?	0.545
14. Is there a plan for re-assembling family members after incidents?	0.671
15. Is there a plan for informing the covered population about how to access biohazard prevention services?	0.699
16. In an emergency, is there a plan for communicating with the media and the general public?	0.705
17. Is there a plan for accessing covered groups within 24–48 hours of the incident to make them aware of available services?	0.594
18. Is there a specific communication network to disseminate and receive information from other CBHOs or social groups?	0.701
19. Is there a plan to work with other health centres to provide care to the affected population at the time of the incident?	0.596
20. Are there plans to facilitate participation in working groups, councils and health committees in the province?	0.592
21. Is the process of getting help from your organisation ensuring the urgent needs of the health system to be met been clarified?	0.666
22. Is there a plan to increase your organisation's human resources in an emergency?	0.669
23. Is there a plan to provide primary healthcare to the covered population?	0.521
24. Is there a plan to report suspected cases of a disease in the covered population to the medical university?	0.571
25. Is there a plan for triage and referral of patients to appropriate treatment zones in coordination with the medical university?	0.633
26. Is there a plan to provide workforces of investigation teams and conduct active surveillance to help the health system?	0.577
27. Is there an agreement with your organisation to finance the human resources needed by the health system in accidents?	0.684
28. Does your organisation have a plan to provide environmental health services to clean the treatment spaces when a biological incident happens?	0.684
29. Is there a plan to overcome the overlaps of your organisation’s resources with other CBHOs during hazards?	0.677
30. Is there a plan to evacuate the affected area by a biological incident?	597.

CBHOs: Community-based Health Organizations

CBHO, community-based health organisation.

Table 2

Factor loadings of the infrastructure field

Items	Factor loading
1. Are there appropriate services and technology to notify the accident to the University of Medical Sciences?	0.612
2. Do you plan to share the information of the covered population with the health system?	0.414
3. Can you determine the specific needs of the covered population in epidemics?	0.489
4. Is your organisation able to provide the necessary facilities for vaccination and mass immunisation?	0.489
5. Can your organisation provide an abundant supply of drugs, vaccine and serum through during emergencies?	0.819
6. Can you provide logistic support of healthcare in accidents (eg, power supply, heating equipment, ventilators and ventilation systems)?	0.784
7. Is there a plan to eliminate the legal barriers of participation when responding to an emergency?	0.730
8. Have you introduced your workforce to help the health system during epidemics (as personnel, volunteers and students)?	0.679
9. Is there an agreement to receive emergency funding from government organisations during emergencies?	0.447
10. Has the health system defined the priorities of resource allocation in epidemics?	0.760
11. Can you provide transportation facilities for affected patients (like ambulance and so on)?	0.786
12. Is there an emergency planning for needs such as food, water, medicine and so on at the time of incidents?	0.680
13. Is it possible to provide health advice in English and Arabic to foreign travellers in epidemics?	0.381

Table 3

Factor loadings of the training field

Items	Factor loading
1. Have your organisation been provided any training and instructions regarding preparedness and response during a biological incident?	0.823
2. Is there a plan to educate the health and clinical members of your organisation to screen, surveil and report cases of biological incidents?	0.603
3. Is there a plan to train members of your organisation to support the mental and moral health of the affected population during emergencies?	0.545
4. Is there a plan to train the members of your organisation to resuscitate patients?	0.614
5. Have you already provided the Department of Health’s guidelines for disaster management to your organisation?	0.705
6. Are there instructions for the covered population to access health services in case of infection?	0.550
7. Have the necessary training and skills to adapt psychologically in emergencies been provided to members of your organisation?	0.733
8. Have epidemic training programmes tailored to target groups (responsible personnel, volunteer citizens and covered population) been considered?	0.706
9. Are there documented lessons learnt from the experience of partnership with the health system in epidemics?	0.791
10. Does your organisation have plans to participate in exercises and drills related to disasters and emergencies?	0.800
11. Is there a plan to train your organisation's members conducting exercises and drills of disasters?	0.748

Table 4

Factor loadings of confirmatory factor analysis

Fitting indexes	scores
Comparative Fit Index	0.921
Tucker-Lewis index	0.918
Root mean square error of approximation	0.052
Weighted root mean square residual	1.158

Reliability analysis

The Cronbach’s α in the planning, training and infrastructure field were 0.938, 0.916 and 0.889, respectively. The Cronbach’s α of the whole tool was 0.944. Scores showed that the reliability of the tool is acceptable.

Concurrent criterion validity

Thirty-four CBHOs completed the Austin26 questionnaire, and the Spearman correlation coefficient was 0.736 that mean the criterion validity of the tool is acceptable.

Discussion

In this study, a tool was developed to assess the preparedness of CBHOs during biohazards. The researcher-made tool consisted of planning, training and infrastructure fields based on previous studies.7 8 27 30 31 After achieving face and content validity, CFA was used for the three-factor model fit. EFA was also used to compare the items replaced in the proposed model.

Some studies noted that items with a factor load above 0.7 and even 0.4 were acceptable.32 33 Therefore, in the CFA, a single item with factor load less than 0.4 was deleted. table 4 shows that the RMSEA value of the model is 0.052. According to previous studies, if the RMSEA is between 0.05 and 0.08, the model is acceptable.34 The index of CFI and TLI is more than 0.9, which shows that the three-factor model is acceptable.35 Hence, in this study, we used CFA to compare the model fit of our tool with the proposed structure of The HHS Office of the ASPR. The office has divided the functions of community preparedness into three categories as planning, training and skills, equipment and technology.27 However, in this research, the research team consented to use the term ‘infrastructure’ instead of ‘equipment and technology’.

According to a cross-sectional study, the most challenging aspect in implementing the 15 capabilities of the CDC’s guideline is training and planning, and 18% of failing is related to infrastructure field.36 37 Evidence from a qualitative study shows that planning and training fields are the most significant challenges faced by health workers in response to Hurricane Sandy.38 Another study showed ‘“community preparedness”’ as the common standard in both the accreditation standards developed by Public Health Accreditation Board (PHAB) and the 15 capabilities of CDC’s guideline. Therefore, strengthening all organisations in community-based preparedness can improve both accreditation standards and follow CDC guideline. Also, the fields of training, planning and infrastructure are common in both the guideline and the accreditation standards.39 In this regard, the categorisation of CBHOs’ preparation activities in the three areas in our research is consistent with the study.

Importance of community partnerships in disasters has motivated researchers to develop various assessment tools for evaluating CBHOs’ preparedness. Glik et al and Clawson et al developed tools as an instrument monitoring collaboration between local health departments and CBOs.8 40 However, they focused more on the duties of health departments in engaging CBOs. Austin et al 26 showed that CBOs’ preparedness in earthquake needs seven clusters of assessment including internal training, external response, response capabilities, information collection and distribution to staff, preparation, building protection and supplies. Moreover, Baezconde et al assessed preparedness of non-governmental organisations (NGOs) considering social and structural needs. Socially, NGOs have high ‘social will’ but little ‘community readiness’ to participate in emergencies. Structurally, NGOs’ linkage to voluntary organisations and public health departments lack enough coordination.41 An assessment tool with biohazard approach for CBO’ preparedness is rare in literature, and we have tried to fill this gap.

The CBHOs in Iran conduct weekly meetings at the health departments of medical universities wherein their fields of cooperation are identified. The representatives of CBHOs participate at these meetings, and health authorities discuss fields that health systems lack an adequate budget or cannot intervene due to legislation. Thus, CBHOs depending on their capabilities would offer their cooperation with the health system. The aforementioned meetings will be held more actively in times of disasters.42 43

The evaluation of these CBHOs in Iran in terms of their capacities, capabilities, strengths and weaknesses in the field of planning, training and infrastructure can be achieved using this assessment tool. Therefore, the government departments that licensed CBHOs could plan out the relevant training needed before biohazards and be aware of their capacities to use them in times of disasters or empower capacities for future actions. Moreover, biological hazard is a threatening disaster in Iran owing to the various cultural and religious mass gathering posing a high risk of occurrence, namely, the epidemic of influenza, hepatitis B and D, various types of haemorrhagic fevers and brucellosis. In response to these epidemics, Iran has used many strategies to cope with them. These include recruiting a surveillance system with mandatory reports of particular disease according to the guidelines of the ministry of health, training health personnel to prevent transmission of the disease in the community especially the high-risk group, using mass media to persuade community involvement in preventing transmission, controlling the vectors by using pesticides, educating people with high-risk job due to exposure to the disease source, preventing high-risk people in participating in Hajj (pilgrimage), mandatory vaccination if participating in certain religious occasions and educating hygiene habits like washing hands and using a mask in very crowded places. CBHOs participate in these actives to serve their covered population.44–48 This tool can help them to assess their preparedness in biohazard and recognise the need of enhancing their capacity.

Limitations

It was not possible to check the reliability through test–retest due to the low participation of CBHOs in the research. We will use some qualitative or mixed method studies to verify this tool in future. In comparison with the ASPR guidelines, our findings might reflect some potential item limitations regarding to ambiguous translation and wording sentences that would make difficult to answer for participants.

Conclusion

We collected data on CBHOs through the ministry of health. We used creditable mathematic calculation based on mature model and revised by experts’ opinion. Besides, this tool is used to measure the preparedness of CBHOs and their ability to participate in biohazard and identify their weaknesses. The tool could aid better understanding of the training and skills required for CBHOs to participate during hazards. Furthermore, CBOs can use this tool to participate in drills and practices. Finally, the preparedness tool can help CBOs improve their planning, training and infrastructure. The authors will verify credibility and extend its usability to improve its quality continuously.