The influence of organizational jurisdiction, organizational attributes, and training measures on perceptions of public health preparedness in Alberta.

OBJECTIVES: Recent international events including the H1N1 influenza pandemic and the rising incidence of West Nile Virus throughout North America have brought critical attention to the Canadian public health system and how prepared the system is to respond to various types of contemporary public health threats. The current work assessed the association of organizational jurisdiction, organizational attributes, and training opportunities with three different measures of public health preparedness in the province of Alberta, Canada.
METHODS: Organizational representatives involved in the delivery of public health systems completed an online questionnaire that asked about organizational attributes and training opportunities available to employees, their perception of organizational preparedness, and their connections to other organizations in Alberta.
RESULTS: Findings revealed that (1) perceived human and material resources preparedness was associated with training opportunities, (2) perceived informational needs was associated with organizational size, and (3) whether an organization exercised their written preparedness plan in 2006 was associated with organizational jurisdiction.
CONCLUSIONS: These findings help fill a gap in the literature with respect to identifying how organizational characteristics are associated with different aspects of preparedness.

Introduction

Recent international and national events such as the H1N1 influenza pandemic, the rising incidence of West Nile Virus throughout North America, and the SARS outbreak in 2002–2003 in Toronto have brought critical attention to the Canadian public health system and how prepared the system is to respond to various types of contemporary public health threats (Moore et al. 2006; Nelson et al. 2007). The public health system is responsible for preventing disease, protecting the health of the population, and promoting health and overall well-being (Canadian Public Health Association 2001). The effective coordination of all agencies at metropolitan, regional, provincial, and federal levels is essential to the management of public health threats (National Advisory Committee 2003).

An examination of the literature yielded several different ways of defining public health preparedness, numerous subjective and objective measures, and different key elements that characterize a well-prepared community (Nelson et al. 2007; Riederer-Trainor et al. 2005; Department of Homeland Security 2007; Federal Emergency Management Agency 2008). The current work used a definition of public health preparedness that combined contributions from the RAND Corporation (Riederer-Trainor et al. 2005), the United States’ Department of Homeland Security (2007), and the Federal Emergency Management Agency (FEMA) (2008). Public health preparedness is defined as a multifaceted planning process designed to protect communities by coordinating and integrating activities to strengthen the ability to prevent, respond to, and recover from major events such as natural disasters, acts of terrorism, infectious disease outbreaks, or other manmade disasters, particularly whose scale, timing, or unpredictability threatens to overwhelm routine capabilities. Preparedness efforts aim to ensure that the public health system is able to mitigate the anticipated effects of illness and injury, limit morbidity and mortality, and sustain societal, economic, and political infrastructure following a public health emergency.

While governmental agencies, private enterprises, and professional associations in the United States have developed numerous measures and instruments for evaluating the preparedness levels of public health agencies and their communities, a paucity of information exists specific to the Canadian context. Moreover, a recent American review by Asch et al. (2005) revealed that of the 27 published population-based instruments for planning or evaluating preparedness that were reviewed, only two were available in the peer-reviewed literature. The current work aims to fill a gap in the literature by examining how different aspects of public health preparedness relate to specific organizational characteristics. This research not only provides information on how changes in organizational elements may impact an organization’s sense of preparedness or their capacity to exercise preparedness plans, but it also serves as the foundation for the development of standardized preparedness assessment tools.

One of the key challenges facing researchers and practitioners in the field of public health preparedness is a lack of standardized measures and metrics which makes it difficult to assess organizational preparedness. Measures often vary from one agency to another and shift from year to year as revisions and improvements are made (Nelson et al. 2007; Asch et al. 2005; Fraser 2007). What results is a series of sometimes contradictory requirements, tools, and ideas of what constitutes preparedness. This makes it challenging for health officials, businesses, non-profit organizations, and the general public to have a clear and thorough understanding of preparedness. Moreover, the dynamic nature of this field makes it difficult to evaluate the effectiveness of programming and ensure that public health agencies remain accountable to important stakeholders.

The current work analyzes public health preparedness in the province of Alberta, Canada, by assessing the association between three different aspects of public health preparedness and various organizational attributes. The study addresses three questions: (1) What is the association between perceived human and material resources preparedness and organizational elements, such as size, training opportunities, and jurisdiction?; (2) What is the association between perceived informational needs and these same organizational elements?; and (3) What is the association between whether an organization exercised its public health response plan in 2006 and these organizational elements?

This work assesses public health preparedness along both subjective and objective dimensions. For the subjective dimension, the study examined organizational perceptions of human/material resources and informational needs. For the objective dimension, the current work examined whether an organization had exercised its written emergency response or public health emergency plan. Emergency response or preparedness plans are generally developed by a small group of individuals within a jurisdiction, while responses to real-world emergencies require cooperation among a large number of organizations and their individual members (Nelson et al. 2007). A plan may not be successful the first time it is executed as a result of uncertainties with respect to which elements will be necessary for a given situation and what unforeseen conditions may occur, requiring adaptations to the written plan (Nelson et al. 2007). These considerations suggest that organizations that have executed their organizational plans either as a practice run or in reality and have engaged in a quality improvement process are more likely to assess their level of public health preparedness higher than those that merely have a written plan.

Methods

Sample description

The current work used data collected through the “Public Health Preparedness and Responsiveness in Alberta: An Inter-Organizational Relations Study of Public Health Preparedness and Response Networks” research project. Using a stratified random sampling technique, 595 organizations geographically spread across four jurisdictional levels (provincial, regional, city, and town/village) were invited to complete an online questionnaire. In an initial census, these organizations were identified as potentially playing a role in public health preparedness or emergency management within the province. Email and regular mail invitations were sent to organizational representatives who were asked to select the most appropriate person in their organization to complete the online questionnaire.

Funding and ethics

This project was funded by the Alberta Heritage Foundation for Medical Research and received ethics approval from the Office of Biomedical Ethics of the Faculty of Medicine, University of Calgary and from the Health Research Ethics Board of the Faculty of Medicine and Dentistry, University of Alberta.

Organizational questionnaire

The online organizational questionnaire was developed specifically for this study to assess public health preparedness in Alberta (Online Resource 1). Organizational questionnaires were completed between November 2007 and April 2008. Organizations self-identified the person best able to complete the questionnaire. The questionnaire asked respondents about the general characteristics of organizations, their ties to other organizations, and perceptions of the institutional environment and preparedness network. Additional details on the survey development and structure have been previously reported (Hall et al. 2010). Using these organizational data, this work evaluates individual organizational effectiveness related to public health preparedness and emergency management. Individual organizational effectiveness is assessed by structural indicators that measure the capacity of an organization for effective performance and evaluate the qualifications of organizational actors to perform the work (Scott 1998). Specific measures of perceived organizational effectiveness include the training and qualifications of the staff, and the adequacy of equipment and facilities. Data on organizational attributes such as size (number of employees and volunteers), budget, date of establishment, and operational sector were also collected.

Measures

Outcomes

Outcome variables were (1) perceived human and material resources preparedness, (2) perceived informational needs preparedness, and (3) the dichotomous indicator of whether an organization exercised a written emergency response or public health response plan in either practice or a real situation during 2006. For perceived human and material resources preparedness, organizational representatives were asked to rate their level of satisfaction with the resources available to their organization in the following areas: (1) availability of personnel trained in public health, (2) availability of personnel trained in emergency management, (3) availability of training programs for staff, and (4) availability of technology and equipment for public health preparedness/emergency management. For perceived information needs, representatives were asked to rate their level of satisfaction with (1) availability of information on regional activities related to public health preparedness, (2) availability of information on other organizations’ public health preparedness activities in the area, (3) availability of information on provincial activities related to public health preparedness, and (4) availability of information on provincial sources of funding. For each item, respondents indicated their level of satisfaction using a four-point Likert scale ranging from very dissatisfied to very satisfied. A total summative score was obtained for each responding organization.

Correlates

The main organizational correlates that were examined were jurisdiction, size, age, principal focus, and level of training. Jurisdiction was divided into four strata: (1) provincial-level departments, (2) sub-provincial administrative agencies, i.e., emergency management districts (disaster services) and regional health authorities, (3) cities, and (4) towns and village areas. These were dummy-coded with the town and village jurisdiction used as the referent. Organizational age was based on the number of years between 2008 and its year of establishment; the principal area of focus was based on whether an organization reported working in emergency management (EM), public health preparedness (PHP), both EM and PHP, or other; and organizational size represented the total number of employees and volunteers that an organization reported working on public health preparedness and/or emergency management activities. Training was based on whether organizational representatives reported having or not having (yes/no) (1) conducted a training needs assessment to identify gaps in employee knowledge, skills, and abilities in relation to public health preparedness; (2) participated in training exercises or courses on public health preparedness or emergency management sponsored by Alberta Health and Wellness, Alberta Municipal Affairs or the Public Health Agency of Canada; and (3) prepared written agency-specific job descriptions that defined knowledge, skills, and abilities needed for emergency roles and responsibilities and conformed with recognized standards for training and certification. If an organization reported having completed at least one of the above, the organization was coded as providing training opportunities.

Statistical analysis

There were two stages to the statistical analysis. First, confirmatory factor analysis was used to determine if the two à priori subjective indicators of public health preparedness, one which addressed human/material resources and the second which addressed informational needs, conformed to expectations using eight survey questions related to organizational perceptions of the environment. A dimensional factor analysis approach was employed. The extraction method was principal components. In the dimensional analysis, the principal factors were extracted from the correlation matrix using communality estimates. Factors were defined with four considerations: (1) factor loadings greater than 0.40, (2) avoidance of redundant terms, (3) scree plot assessment (eigenvalues greater than 1 (Kaiser-Guttman rule) and falling within the sharp descent part of the plot before the eigenvalues start to level off), and (4) structure simplicity. This approach was consistent with that of Dorn et al. (2007). Factors were rotated using a varimax rotation procedure.

Second, multiple linear and logistic regression models were used to test the association of the three preparedness outcomes with organizational characteristics. Three sets of models were examined: (1) human and material resources, (2) informational needs, and (3) having exercised a preparedness plan in 2006. In the first set of models, perceived human and material resources preparedness was regressed on organizational jurisdiction, age, the principal area of focus, size, and organizational training measures. The annual budget variable was not included in the regression models as organizational size was used as a proxy indicator of an organization’s annual budget. Organizational variables were then removed from the previous model if they failed to reach a significance level of p < 0.10 for the specific outcome. The regression model was then re-run with only significant organizational variables remaining (jurisdiction, number of employees and volunteers, and training measures). Overall model significance was assessed. This same modeling strategy was repeated in examining the two other preparedness outcomes, except that for the logistic regression model, fit was evaluated using the Hosmer–Lemeshow Goodness-of-Fit Test. In addition, the model’s accuracy in discriminating between those organizations that did or did not exercise their plan was evaluated using a Receiver Operating Characteristic (ROC) curve. Statistical analyses were performed using SPSS, version 16.0.

Results

Response rates and demographics

Of the 595 organizational representatives invited to complete the organizational questionnaire, 125 responded with complete information for the analyses in the current investigation. Several departments within larger organizations, such as those at the provincial or regional levels, chose to respond as part of that larger organization or agency. At the town level, many organizations that were asked to participate did not see their organization’s mission as concerning public health preparedness and chose not to respond. In terms of respondent coverage, the response percentage at each jurisdictional level was as follows: provincial (45.2%), regional (37.9%), city (30.7%), and town/village (20.9%). Despite the low overall organizational-level response rate, respondent data are geographically and jurisdictionally spread across agencies involved in public health preparedness. For example, in terms of geographical sample coverage, 87.5% of Alberta cities and 69.1% of the towns and villages sampled provided at least one organizational respondent. In 68.8% of the cases at the city level and 59.5% of the cases at the town/village level, respondents represented fire department or disaster service organizations. Descriptive statistics outlining the principal area of organizational focus, jurisdictional level, organizational size, and annual budget are reported in Table 1.

Table 1

Alberta public health preparedness sample: descriptive statistics (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Variable	Percentage (%)
Organizational principal area of focus
Emergency management (EM)	37.6
Public health preparedness (PHP)	4.8
EM/PHP	21.6
Other	36.0
Jurisdictional level of organizations
Provincial	22.4
Regional	19.2
City	24.0
Town/village	34.4
Exercised preparedness plan in 2006	48.2

Perceived preparedness scores

Principal components analysis confirmed a two-factor solution (which accounted for 62.3% of the total variance) which was parsimonious, had good, simple structure, and could be meaningfully interpreted. Factor loadings were consistent with the fact that we anticipated that organizational satisfaction with human and material resources would be distinct from satisfaction concerning informational needs. The rotated solution, as shown in Table 2, yielded two interpretable factors. The perception of human/material resources preparedness factor accounted for 46.5% of the total variance and loaded the highest on the four items reported previously. The perception of informational needs preparedness indicator accounted for 15.8% of the total variance and loaded the highest on the four other items reported as assessing perceived informational preparedness.

Table 2

Factor loading scores of perceived resource items for subjective public health preparedness measures (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Items	Factor loadinga
Human/material resources
Availability of personnel trained in public health	0.44
Availability of personnel trained in emergency management	0.76
Availability of training programs for staff	0.84
Availability of technology and equipment for PHP/EM	0.76
Informational needs
Availability of information on regional activities related to PHP/EM	0.78
Availability of information on other organizations’ PHP/EM activities in your area	0.78
Availability of information on provincial activities related to PHP/EM	0.89
Availability of information on provincial sources of funding	0.60

aLoadings following varimax rotation

Multiple regression analyses

In the first set of analyses, perceived human and material resources preparedness was found associated with only an organization’s level of training opportunities (B coefficient (B) = 0.29, p value (p) = 0.01). Removing the non-significant predictor variables from the full model yielded a significant bivariate regression model (F (1,83) = 5.99, p = 0.02). Training opportunities (B = 0.26, p = 0.02) remained significantly associated with perceived human and material resources preparedness. Results of the multiple regression and bivariate analyses are shown in Table 3.

Table 3

Multiple regression analysis of the human/material resources indicator on organizational jurisdiction, size, and training measures (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Variable	Full model coefficientsa	Partial model coefficientsa
Training opportunities	1.46* (0.56)	1.22* (0.50)
No training opportunities (referent)	–	–
Organizational size	0.00 (0.00)	–
Jurisdiction—city	0.37 (0.29)	–
Jurisdiction—regional	−0.29 (0.32)	–
Jurisdiction—provincial	0.32 (0.32)	–
Jurisdiction—town/village (referent)	–	–
Constant	−1.16 (0.49)	−1.50 (0.55)

aStandard errors in parentheses

* p < 0.05

In the second set of models, organizational size was alone significantly associated with perceived informational needs preparedness (B = 0.004, p < 0.01). Removing the non-significant organizational variables from the model yielded a significant bivariate regression model (F (1,75) = 5.87, p = 0.02). Organizational size remained significant (B = 0.004, p < 0.01). Results of this set of models are shown in Table 4.

Table 4

Multiple regression analysis of the informational needs indicator on organizational jurisdiction, size, and training measures (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Variable	Full model coefficientsa	Partial model coefficientsa
Training opportunities	0.48 (0.57)	–
No training opportunities (referent)	–	–
Organizational size	0.004* (0.001)	0.004* (0.001)
Jurisdiction—city	0.38 (0.29)	–
Jurisdiction—regional	−0.32 (0.32)	–
Jurisdiction—provincial	0.26 (0.33)	–
Jurisdiction—town/village (referent)	–	–
Constant	−0.62 (0.56)	−0.09 (0.12)

aStandard errors in parentheses

* p < 0.05

In the final set of analyses, an organization’s likelihood of having exercised a written emergency response or public health response plan in 2006 was found associated with organizational jurisdiction. Compared with town/village-level organizations, city- (odds ratio (OR) = 11.66, 95% confidence interval (95%CI) = 2.82–48.32) and regional-level organizations (OR = 7.96, 95%CI = 1.40–45.31) were more likely to report having exercised their response plan. Provincial agencies were not significantly more or less likely to report having exercised their response plan than town organizations. Other organizational characteristics were not significantly associated with having exercised the response plan. For the final model, only the jurisdictional variable was thus retained. Respondents at the city level remained more likely to have exercised an emergency response plan than those at the local level (OR = 11.43, 95%CI = 3.09–42.27). Similarly, respondents at the regional level were more likely to have exercised an emergency response plan than those at the local level (OR = 7.14, 95%CI = 1.65–30.88). The Hosmer–Lemeshow Goodness-of-Fit Test indicated that the fit of the model was adequate (p > 0.10). The area under the ROC curve, 0.75, indicated good discrimination (Fig. 1). Results of the multiple logistic regression analyses are shown in Table 5.

Fig. 1

Receiver Operating Characteristic (ROC) curve for the multiple logistic regression analysis of preparedness on organizational jurisdiction, size, and training measures (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Table 5

Multiple logistic regression analysis of exercising a preparedness plan on organizational jurisdiction, size, and training measures (Alberta, 2008, Public Health Preparedness and Responsiveness in Alberta)

Variable	Full model odds ratioa	Partial model odds ratioa
Training opportunities	1.84 × 109 (1.47 × 109–2.21 × 109)	–
No training opportunities (referent)	–
Organizational size	1.00 (0.99–1.01)	–
Jurisdiction—city	11.66 (2.82–48.32)	11.43 (3.09–42.27)
Jurisdiction—regional	7.96 (1.40–45.31)	7.14 (1.65–30.88)
Jurisdiction—provincial	3.34 (0.72–15.39)	3.57 (0.98–12.97)
Jurisdiction—town/village (referent)	–
Constant	0.00	0.28

* p < 0.05

a95% confidence intervals in parentheses

Discussion

Present inconsistencies in how public health preparedness is measured from one agency to another prompted the current work to assess how three different possible (and clear) measures of preparedness were related to organizational attributes. These measures included two subjective indicators, one on perceived human/material resources and the other on perceived informational needs, and one objective indicator of whether an organization had exercised a preparedness plan in either practice or a real situation during 2006. These three items were each associated with a single organizational variable: namely training opportunities, organizational size, and organizational jurisdiction, respectively.

The study has shown that organizational characteristics are associated with different dimensions of organizational-level preparedness in Alberta. Rather than finding a blanket association of a specific organizational characteristic with each dimension of preparedness, however, the study shows that organizational characteristics may be uniquely associated with different aspects of preparedness. The results indicate that consideration of a range of organizational attributes may be a critical component of preparedness evaluations since the significance of a given organizational characteristic is not the same across different aspects of public health preparedness. Moreover, these findings help fill a gap in the current literature with respect to which aspects of preparedness relate most strongly to organizational features.

Each finding supports to some degree recent recommendations on building public health preparedness capacity. The association between organizational training opportunities and perceived human and material resources preparedness is consistent with the Naylor Report’s statement in Canada that accessible and effective training programmes in a number of formats for staff and personnel are essential for maintaining a public health system capable of responding to contemporary public health threats (National Advisory Committee 2003). The National Center for Disaster Preparedness at Columbia University’s Mailman School of Public Health indicates that many individuals within the public health preparedness workforce do not have the necessary knowledge and skills for full participation in planning, response, and evaluation activities as reported by public health leaders (2008). Consequently, they recommend competency-based emergency preparedness training that is effective, efficient, and economical for all individuals involved in the sector. Multiple works have also reported that inadequate training is a barrier to improving levels of preparedness (Nelson et al. 2007; Wasserman et al. 2006). As suggested in this study, the presence of training opportunities do appear to be significant ingredients in how organizations perceive their degree of preparedness in terms of human and material resources.

The association between organizational size and perceived informational needs preparedness suggests that organizations with greater personnel committed to public health preparedness or emergency management functions are more likely to report having adequate information related to public health preparedness training activities and funding in comparison with smaller organizations. The finding is logical, but also highlights the importance of having adequate personnel available to prepare for and manage the informational demands that arise at a time of an emergency. Several studies have indicated that not having enough personnel dedicated to planning and responding to emergencies is a significant barrier to preparedness both within organizations and across jurisdictions (Nelson et al. 2007; Wasserman et al. 2006; PHEP 2008). For the Alberta sample, the fact that perceived informational needs and not perceived human and material resources preparedness were associated with organizational size suggests that organizations may make a careful distinction between the number of personnel available to them and their personnel’s level of training. Greater organizational size may contribute to an organization’s capacity to access and manage information about provincial and regional activities, but does not appear in this sample to be an indicator of the availability of trained personnel.

Finally, the association between organizational jurisdiction and whether an organization had exercised its plan suggests the possible presence of cross-jurisdictional disparities in preparedness. In the Alberta sample, organizations at the city and regional levels were significantly more likely to report having exercised their plans than those at the town/village level. While a paucity of information related to jurisdictional influence on preparedness levels exists, the current results mirror findings from recent work in Florida that has examined emergency management networks. According to this work, smaller municipalities have primarily relied upon counties or regional network supports for disaster-related functions since higher jurisdictional levels are better prepared for managing emergencies (Caruson et al. 2008). Other US research has indicated that aside from major metropolitan areas, few counties, cities, or towns have the capacity to respond to public health emergencies independently (Koh et al. 2008). A US nationally representative survey conducted from 2000 to 2001 found that only 25% of local public health jurisdictions indicated they could deliver at least 60% of essential public health services in the case of a terrorist event (Baker and Koplan 2002). At lower jurisdictional levels, limited resources and trained personnel may limit levels of preparedness. For example, local agencies may not have access to laboratory services which are used to identify organisms and environmental agents which affect community health, thus impairing their readiness to respond to an infectious disease outbreak or biological warfare event. Under these circumstances, it is imperative that local organizations develop formal relationships with laboratories outside the jurisdiction to assure services are available if needed (Centers for Disease Control and Prevention 2002).

One limitation of this study concerns the representativeness of the sample. The sampling frame and design was meant to be as comprehensive as possible and cover Alberta geographically; some organizations, particularly town/village departments and sub-agencies, did not respond. The total sample of 595 organizations invited to complete the organizational questionnaire included bureaus and departments within larger umbrella organizations or jurisdictions. On many occasions, sub-agencies chose to respond as part of the umbrella organization even though each has a separate role to play should a real public health emergency arise. For example, within Alberta Health and Wellness, the “Emergency Health Services” department and the “Public Health Division” were asked to complete the online survey, but they collaborated and completed one questionnaire. In addition, because the study cast a wide net of organizational types including those who would only be peripherally involved in preparedness activities, there may have been a number of organizations that chose not to respond since they did not see preparedness as central to their organizational mission.

A second limitation concerns the preparedness measures themselves. Ideally, performance measures should consist of three different dimensions: structure (capacity), process, and outcomes (Donabedian 1978). The organizational questionnaire focused primarily on structural measures of preparedness as these are the most amenable to policy changes. In terms of process, the questionnaire assessed if organizations had a written plan or had exercised their written emergency response or public health emergency plan. Since process measures are more proximally related to outcomes than structural ones, future work might be strengthened by placing greater emphasis on process measures related to preparedness (Asch et al. 2005). For example, questions asking how well the workforce is performing their preparedness duties (process) rather than asking if the workforce is provided with training opportunities (structure) would be beneficial. The current work does not include any outcome measures. Examining outcomes as a means of evaluating public health preparedness is challenging as public health emergencies are rare and the averted morbidity and mortality is difficult to ascertain (Asch et al. 2005).

Flexible and resilient systems must be in place to be prepared for and respond to unprecedented disasters and epidemics of varying scope and severity. According to Dorn et al. (2007), measures that reliably assess system strengths and weaknesses are necessary to bring about changes to improve system weaknesses before they become a liability during a disaster or other public health event. Yet, little is known comparatively about how prepared organizations either view themselves or are in terms of objective markers. The lack of a preparedness metric in this regard places important limitations on capacity-building policies. In examining the association between organizational characteristics and multiple dimensions of preparedness, the current work takes an important step in building a preparedness metric and linking it to organizational features. Since the measures examined in the current work are each associated with a different organizational characteristic, it may be important to include all three as part of a standardized preparedness metric. The development and use of such indicators to assess overall system preparedness as well as levels of preparedness would help address a gap in the current public health emergency surveillance and response system. This is potentially significant since these measures may better allow health officials, business and non-profit organizational members, and the general public to have a clear and thorough understanding of preparedness while increasing accountability to important stakeholders.

Below is the link to the electronic supplementary material.

Online Resource 1 (PDF 77 kb)