Workforce Competencies in Syndromic Surveillance Practice at Local Health Departments.

CONTEXT: As the science and practice of syndromic surveillance (SyS) evolve, it has increasing utility for public health surveillance at the local level. Local health departments (LHDs) require specific organizational and workforce capabilities to use SyS data. In 2013, more than half of the LHDs reported using SyS, although little has been reported about LHD workforce capabilities in SyS.
OBJECTIVE: To conduct an assessment of self-reported knowledge and skills in SyS tasks to effectively target technical assistance to different levels of LHD need. DESIGN, SETTING, AND PARTICIPANTS: A stratified sampling design based on LHD jurisdiction population and SyS status was employed. Data were drawn from the 2015 Biosurveillance Needs Assessment Survey, which captured variables related to LHD use of SyS, management of systems, and self-reported proficiencies in a typology of SyS functionalities developed by a workgroup of subject matter experts in SyS. Respondents were US-based LHD public health practitioners. Estimation weights were applied during analysis to determine the national representation of the responses. MAIN OUTCOME MEASURES: Respondents self-reported proficiency in 26 SyS tasks within 5 categories, analyzed by LHD jurisdiction size and respondents' years of SyS experience.
RESULTS: SyS expertise varied widely across LHDs. Less than 50% of workers who have access to SyS demonstrated overall proficiency within any of the task areas: communication, data use, data analysis, quality monitoring and assurance, and system design and development. SyS users were strongest in data use tasks. Proficiency in SyS practice corresponded directly with respondents' years of SyS experience and the LHD jurisdiction size.
CONCLUSION: SyS practitioners display a wide range of proficiencies both within and across SyS tasks. Considerable gaps in proficiencies of all areas of SyS practice indicate a need for technical assistance and knowledge dissemination to improve SyS practice as an important component of an LHD surveillance strategy.

Local health departments (LHDs) are at the front line for the prevention, detection, response, and mitigation of public health threats in their communities. Syndromic surveillance (SyS) is an approach that turns electronic health and health-related data into information that is timelier than that available through traditional surveillance methods.1 Built to meet the rapid need for population-level data in the event of an emergency, notably a bioterrorist attack, SyS is now being applied to the detection and monitoring of infectious disease trends and outbreaks, chronic disease exacerbations, injuries, drug overdoses, and a broad array of other areas of public health surveillance.2–6 SyS is a health informatics functionality that involves the use of automated data acquisition, complex analytic tools, and alerting technologies that require specific workforce competencies.7

SyS has unevenly been adopted into surveillance practice among local and state-level health departments.6,8,9 The 2013 National Profile of Local Health Departments conducted by the National Association of County & City Health Officials (NACCHO) found that 62% of LHDs use an electronic SyS and that LHDs representing larger populations were more likely to use SyS.10 Another study reported that 66.5% of LHDs use SyS and described a similar association with LHD jurisdiction size.7

The use of SyS is closely linked to organizational and individual capabilities in a range of disciplines from epidemiology and statistics to informatics, computer programming, and communications. Workforce capabilities have been shown to be associated with LHD capacity in public health informatics.11

As the practice of SyS expands to include all-hazards monitoring, the data and information provided have increasing utility for building surveillance capabilities at LHDs. As the role of public health informatics grows for public health agencies, the need for improved workforce development in SyS also expands.12,13 In 2015, to further understand how LHDs use and manage their SyS systems and SyS workforce capabilities, the International Society for Disease Surveillance (ISDS), in collaboration with NACCHO, conducted the Biosurveillance Needs Assessment Survey (BNAS), to identify and measure LHD biosurveillance system capabilities and workforce competencies nationwide. This study describes levels of proficiency across a range of SyS functionalities and patterns across LHDs of different jurisdiction size. The results identify gaps in knowledge and skills that can be addressed to improve both individual and organizational capabilities in real-time public health surveillance.

Methods

Data and sampling design

The data for this study were taken from the 2015 BNAS conducted by ISDS in collaboration with NACCHO. A BNAS workgroup consisting of 14 experts with experience in SyS from local, state, and federal agencies provided subject matter expertise throughout the project.

A sample of 500 LHDs was selected from the NACCHO 2013 National Profile of Local Health Departments database of approximately 2532 LHDs using a stratified sampling design based on 3 population strata (<50 000, 50 000-499 999, and 500 000+) and LHD SyS status (yes, no, no response) (see the Table, Supplemental Digital Content 1, available at: http://links.lww.com/JPHMP/A241, which describes the BNAS sampling plan). LHDs that represented populations with more than 500 000 people were systematically oversampled to ensure a sufficient number of responses from large LHDs for the analysis. Statistical weights were applied to the analysis to account for the oversampling and the disproportionate response from LHDs of different size jurisdictions.

The target respondent at jurisdictions that used SyS was one person from the LHD who was most familiar with the use of SyS at the LHD and/or the concept of electronic emergency department SyS. If the jurisdiction did not use SyS, the target responder was the person who had direct responsibility for epidemiology and disease surveillance functions. The final survey was piloted by 5 LHDs before being revised and opened for online responses for 8 weeks. Because only aggregated data and no personally identifiable information are reported, approval from an institutional review board was not required.

Measures

The BNAS Web-based survey tool was designed using the SurveyMonkey platform to assess SyS practice and to evaluate self-reported proficiencies in tasks associated with SyS practice. Through a literature review and an iterative, consensus-building review process among the BNAS workgroup and project team, an empirically based typology was created that classified SyS functions into 5 general areas: communication, data use, data analysis, quality assurance and monitoring, and system design and development. Survey respondents who indicated that their LHD had access to SyS data or information were asked to complete a self-assessment of their ability to conduct 26 SyS tasks within the 5 areas.

Only jurisdictions that indicated they had access to SyS were included in the analysis of proficiencies. Self-reported capabilities in each area were the primary measures of interest. For each SyS functionality, the response choices were as follows: (1) “I have never done this”; (2) “I can do this with assistance from others”; (3) “I am comfortable doing this independently”; and (4) “I am confident doing this and providing assistance to others.” For this study, proficiency was defined by 3 or 4 responses, consistent with Bloom's taxonomy of learning.14

Analysis

We describe the use of SyS and self-reported proficiencies in a new typology of tasks associated with SyS practice by LHD jurisdiction size and years of worker experience. Using frequencies and percentages, results were described for workers in 3 LHD jurisdiction sizes (<50 000, 50 000-499 999, and 500 000+ people) and in 3 levels of experience (<3 years, 3-7 years, and >7 years). Although the responses were weighted by 7 population sizes for greater accuracy (<25 000, 25 000-49 999, 50 000-99 999, 100 000-249 999, 250 000-499 999, 500 000-999 999, and 1 000 000+), they were aggregated into the 3 LHD populations sizes typically described in the NACCHO National Profiles. The categories for years of experience were created by the BNAS workgroup on the basis of their subject matter expertise regarding SyS practice, especially in light of the timing of reforms (eg, Meaningful Use) and the availability of a new centralized SyS system (ie, National Syndromic Surveillance Program/BioSense).

Results

LHD use of SyS

The overall response rate was 45%. The final data set represented 226 jurisdictions from 31 states in all 10 health and human services (HHS) regions.

To understand the diversity of SyS use among LHDs, respondents were asked a series of questions to determine whether they have access to SyS data or information from any source (eg, their own system, a state system, or any other source) and specifically whether they manage their own SyS system.

Forty-eight percent of all respondents indicated that they had access to SyS as part of their surveillance efforts. Access to SyS data or information was correlated with the size of the LHD population served; 39.6% of LHDs serving fewer than 50 000 people, 56.4% of LHDs serving 50 000 to 499 999 people, and 81% of LHDs serving more than 500 000 people reported having access to SyS.

Table 1 indicates that LHDs serving populations of more than 500 000 people were also more likely to manage their own SyS system. It is also interesting to note that some respondents in each population category did not know how their SyS systems were being managed.

TABLE 1 •

LHD SyS System Management, by Population Served

Population Served	Access to SyS—Do Not Manage Own System, %	Access to SyS—Do Manage Own System, %	Access to SyS—Not Sure of System Management, %
<50 000 (n = 82)	81.1	7.6	11.3
50 000-499 999 (n = 45)	83.7	12.6	3.7
500 000+ (n = 7)	44.2	50.0	5.8

Abbreviations: LHD, local health department; SyS, syndromic surveillance.

Proficiency across areas of SyS practice

There was a wide variability in self-reported ability to conduct SyS tasks within the different task categories. In addition, respondents who indicated they were proficient in some SyS tasks and categories were beginners in others.

In general, proficiency in SyS tasks was low; less than half of the respondents reported proficiency in 23 of the 26 tasks (Table 2). The data use category had the highest mean self-reported proficiency, although all of the categories had an average proficiency under 50%.

TABLE 2 •

Summary of LHD Worker Self-rated Proficiency in SyS Tasks

SyS Functions (n = 134)	% Proficient
Data use tasks
Identifying events of public health importance	60.3
Interpreting syndromic data and trends	51.8
Working with local syndromic data providers for follow-up of detected cases or events	51.6
Developing procedures for follow-up	49.9
Adapting SyS system for use during a special event/mass gathering/disaster	20.2
Mean percent proficiency in data use tasks	46.7
Communication tasks
Communicating the results of a SyS analysis to various audiences	45.3
Communicating the vision and value of SyS to jurisdiction stakeholders or executive leadership	40.1
Sharing best practices in SyS with colleagues	30.2
Mean percent proficiency in communication tasks	38.5
Data analysis tasks
Managing syndromic surveillance data	43.9
Preparing data visualizations for reports (eg, charts, tables)	41.9
Conducting routine SyS	37.7
Responding to specific analysis requests	31.3
Moving syndromic data across platforms (SAS, SPSS, Excel) for broader analysis and utility	18.9
Mean percent proficiency in data analysis tasks	34.7
Quality assurance and monitoring tasks
Maintaining connections with data providers	33.5
Maintaining processes for ensuring valid system functioning	12.5
Maintaining log of data quality problems	9.4
Troubleshooting system when data gaps or system outages occur	6.7
Establishing system performance indicators	6.2
Mean percent proficiency in quality assurance and monitoring tasks	13.6
System design and development tasks
Getting buy-in and support from LHD executive leadership	38.3
Implementing standardized SyS syndrome definitions	17.6
Developing new SyS syndrome definitions for your LHD	13.4
Developing a SyS strategic plan	12.0
Identifying new features needed or updates for the SyS system	10.4
Onboarding data providers to meet Meaningful Use requirements	8.8
System testing and validation	4.3
Developing system requirements and specifications	4.0
Mean percent proficiency in system design and development tasks	13.6

Abbreviations: LHD, local health department; SyS, syndromic surveillance.

The respondents were asked to report the number of years they have personally been conducting SyS at their current health department or any other health department. Self-reported proficiencies were compared with respondents' years of experience conducting SyS (Table 3). In all of the SyS categories, average proficiency was highest among respondents with greater than 7 years of SyS experience. However, for many of the tasks, even the more than 7 years' category had less than 50% of respondents indicating proficiency.

TABLE 3 •

Mean Percent Proficiency in SyS Categories by Respondents' Years of SyS Experience

SyS Categories	<3 y (n = 44), %	3-7 y (n = 50), %	>7 y (n = 41), %
Data use	36.1	42.1	63.4
Communications	26.2	30.9	60.6
Data analysis	27.5	33.3	43.9
Quality assurance and monitoring	11.1	10.7	19.9
System design and development	13.2	10.1	18.1

Abbreviation: SyS, syndromic surveillance.

Similarly, when proficiency across the different SyS categories was compared with the size of the populations served by the LHDs, it showed that average proficiency was highest among respondents who work at LHDs that represent larger populations (Table 4).

TABLE 4 •

Mean Percent Proficiency in SyS Tasks by Size of Population Served

SyS Categories	<50 000 (n = 82), %	50 000-499 999 (n = 45), %	500 000+ (n = 7), %
Data use	39.3	57.0	67.7
Communication	29.6	51.5	58.6
Data analysis	26.8	43.8	67.7
Quality assurance and monitoring	12.4	15.4	16.7
System design and development	9.0	19.7	27.8

Abbreviation: SyS, syndromic surveillance.

Discussion

Conducting SyS and using health data to inform decision making require knowledge and skills in a broad spectrum of activities that span data collection and analysis to information creation and application. The BNAS assessment was designed to identify gaps in the knowledge and skills needed to conduct SyS to better target the type and topic of technical assistance to improve LHD surveillance practice. Because of underlying factors that influence the use of electronic health records (EHRs) for surveillance, a “one-size-fits-all” approach would not be the most effective for improving SyS performance.

While assessments of trends in the EHR use showed slow adoption and risks of discontinuation between 2010 and 2013,7 the Centers for Disease Control and Prevention's efforts to expand and improve the BioSense Platform for SyS, fund jurisdictions to develop SyS systems, and the launch of a National Syndromic Surveillance Program have together revitalized a community of practice focused on SyS.

Conducting SyS and translating health data to public health information for decision making require knowledge and skills in a broad spectrum of activities that span data collection and analysis to information creation and application. Assessing self-reported level of confidence doing different tasks within SyS functional areas indicates where technical assistance activities are needed to increase knowledge and skills. This assessment can also be used to identify subject matter experts and possible peer mentors. In addition, these areas of need have implications beyond technical assistance for LHDs; they are also indicators of areas where schools of public health should focus training for epidemiology students to better prepare them for surveillance practice.

Almost half of all LHDs that responded to BNAS have access to SyS and most of the large LHDs reporting (81%) use SyS. Conversely, only 40% of small (representing <50 000 people) LHDs indicated that they have access to SyS. Although the exact percentages of this study sample reporting use of SyS are different from those reported in the NACCHO 2013 Profile10 (<50 000: 40% vs 57%; 50 000-499 000: 56% vs 69%; >500 000: 81% vs 82%; all: 48% vs 62%), the trend of greater SyS use among larger LHDs was consistent. The current survey is restricted to SyS using emergency department data, which may contribute to the differences, as well as the sampling frame, the person completing the survey, and other methodological and practical factors.

The guidance for targeted technical assistance and the training guidance provided by the results are not straightforward and must be aligned with actual SyS practice at a particular LHD. Indicating, “I have never done this,” does not translate directly to a need for training because the SyS tasks are not equally important to or required by all LHDs. For example, LHDs that get data and information from the state health department and do not manage their own systems might not need training in SyS system monitoring until they are ready to expand their system utility. The results do, however, show that for practitioners who do manage their own system, the areas of greatest need are quality assurance and monitoring and system design and development.

Figure 1 represents the 5 SyS task categories as a progression from basic to advanced SyS practice and reflects beginner to more experienced individual proficiencies. On the basis of these results, basic SyS practice includes proficiencies in data use, communication, and data analysis, whereas quality assurance and monitoring and system design and development tasks are more advanced.

FIGURE •

SyS Categories by Practitioner Experience and Local Health Department Stage of Practice.

As is evidenced by the results, LHDs with advanced levels of SyS practice require the most technical assistance in quality assurance and monitoring and system design and development tasks. The profile of LHDs with advanced levels of SyS includes LHDs that manage their own SyS system, so the quality assurance and monitoring and system design and development tasks are key areas for focusing technical assistance as these are especially pertinent to managing a SyS system.

Conclusion

The results of this study confirm that SyS practice and proficiency in SyS tasks vary considerably among LHDs and surveillance workers. This can be attributed to the way in which surveillance systems, processes, and methodologies have traditionally been created by LHDs to meet their specific needs. The launching of a new BioSense Platform as part of the National Syndromic Surveillance Program, policies such as Meaningful Use to incentivize the use of EHRs, and a growing desire to be able to share data to create local, regional, and national pictures of emerging public health events have led to greater recognition of the need for data standards and interoperability.

To be effective, technical assistance to improve LHD capabilities in SyS practice should be tailored to helping LHD needs while moving toward proficiency in a common and core set of SyS functions. A plan for LHDs to achieve standards in SyS practice and core workforce competencies could help LHDs better integrate SyS into their surveillance systems.

Limitations of the study

Although this survey collected responses from a nationwide sample, the results cannot be extrapolated such that responses from an LHD in one state or HHS region are necessarily indicative of practice at another LHD in that geographic area. This is because of the small number of responses within a given state or HHS region and the tremendous diversity of biosurveillance practice. However, it was possible to draw correlations in practice based on 3 LHD population size categories, where consistent trends were observed.

In all of the SyS categories, greater levels of proficiency were correlated with working at a larger LHD and having more years of SyS experience. While there may be some correlation between LHD size and years of experience, the low number of respondents in the different categories prohibited a detailed analysis.

A number of survey respondents indicated “not sure” about whether they had access to SyS data and information. These responses were most common in LHDs representing small populations and suggest that the survey did not always reach the right person or that SyS is an approach not fully understood within some LHDs. Differences in some duplicate submissions also suggest that there may be some issues of reliability in the responses.

Limitations of this study also include the diversity of surveillance professionals who responded to the survey. The assignment of surveillance responsibilities varies widely across LHDs. Although the survey respondents were public health professionals who could be broadly characterized into executive leadership and public health staff, they represented a range of job titles and responsibilities. This undoubtedly had an influence on the overall intra- and interreliability of the data provided. This also illustrates the complexity of targeting resources and activities to enhance capabilities in surveillance to the appropriate representative/staff member at an LHD or to a particular group by job title.

In addition, respondents self-reported their proficiency in SyS tasks, which was not otherwise verified. The small sample size and the diversity of SyS practice across LHDs also tempered the validity of extrapolation to the national representativeness of the data. The response options “I have never done this” did not include a place to indicate whether the task in question fell under the respondent's duties at his or her LHD. Therefore, a response of “I have never done this” may simply mean that the task is not crucial to the respondent's job rather than indicating a need for technical assistance.