Experience of using information systems in public health practice: findings from a qualitative study.

OBJECTIVE: Data collection and management by local health departments (LHDs) is a complex endeavor, complicated by system level and organizational factors. The purpose of this study was to describe the processes and use of information systems (IS) utilized for data collection, management, and sharing by LHD employees.
METHODS: We interviewed a purposive sample of 12 staff working in the key public health practice areas of communicable disease control, immunizations, and vital records from three LHDs in different states. Our interview questions addressed job descriptions, daily activities, and the use and perceptions of both data and IS in support of their work. A content analytic approach was used to derive themes and categories common across programmatic areas.
RESULTS: Local public health involves the use of mix of state-supplied and locally implemented IS supported by paper records. Additionally, each LHD in this study used at least one shadow system to maintain a duplicate set of information. Experiences with IS functionality and the extent to which it supported work varied by programmatic area, but inefficiencies, challenges in generating reports, limited data accessibility, and workarounds were commonly reported.
CONCLUSIONS: Current approaches to data management and sharing do not always support efficient public health practice or allow data to be used for organizational and community decision making. Many of the challenges to effective and efficient public health work were not solely technological. These findings suggest the need for interorganizational collaboration, increasing organizational capacity, workflow redesign, and end user training.

Introduction

Public health services delivery depends on data and information for enumeration and reporting as part of disease surveillance [1], preventive and medical service delivery [2], local decision making, strategic planning, and quality improvement [3-5]. Such activities make reporting and managing data substantial portions of local public health practitioners daily job activities [6]. However, local public health practitioners work within systems and organizations that can present challenges to the effective collection, management, and sharing of data [7].

Data collection and management in public health practice is a complex endeavor. These efforts require exchanging data with multiple organizations due to overlapping jurisdictional boundaries, shared responsibilities, and mobile populations [7-10]. Generally, increasing the number of data sources increases challenges around maintaining information quality [11]. Additionally, each LHD is often home to a variety of different data management approaches [12], which can include multiple or programmatic-specific information systems (IS) that are not capable of electronically sharing information in a standards-based structured fashion, or reliance on a combination of paper and IS [13-17]. This too creates complications as increasing the number of IS which an individual must use increases the complexity of work and negatively affects productivity [12]. Also, that insufficient or non-interoperable IS can have negative effects on the ability of public health organizations to effectively plan, respond in a timely manner to events, or operate efficiently [18-20]. In fact, evidence indicates the inability to electronically transmit or receive data is the norm for many public health activities [21].

The interaction of individual need for information and presence of a complex arrangement lead us to enrich our understanding of LHD data use, management, and IS characteristics through insights from the lived IS experiences of public health practitioners. We specifically framed our investigation in the context of data gathered and shared between LHDs and state health agencies (SHA) in order to focus on the use of information that is a product of the entire public health system. We selected the activities of immunization delivery, communicable disease control, and vital records as they are performed by a majority of LHDs [17], require data gathering and sharing by multiple public health entities, and these program areas are subject to structural barriers to information sharing [21]. We specifically sought to characterize the perceived IS needs and barriers, as well as IS uses and work-around solutions to accomplish the program goals.

Methods

The qualitative study design involved a purposive sample of LHD employees and open-ended interviews. A content analytic approach was used to derive themes and categories common across programmatic areas.

Sample

In mid-2012, one member of the research team (A1) interviewed 12 staff working in the areas of communicable disease control (n=4), immunizations (n=4), and vital records (n=4) from three LHDs. The three sites were each in different states and included an urban, a rural, and a suburban LHD (Table 1). The sites were selected through our existing organizational contacts with the only requirement that they conducted all three public health activities. Informants had the various titles of epidemiologist, program coordinator, nurse, manager, or registrar. We purposefully interviewed those who were responsible for gathering and data sharing.

Table 1

Characteristics and information systems in use at interviewed local health departments by programmatic area.

		Data Management or Information Systems (IS) Used1
Health department	Number of staff interviewed	Communicable Disease	Immunization	Vital records
Rural	3	Electronic LabsPractice management2Paper records	IIS3Practice management	VRIS4Paper recordsSpreadsheets
Suburban	4	Surveillance IS5STD ISLocal databaseSpreadsheetsPaper records	IIS3MCH IS6Practice managementPaper records	VRIS4Paper recordsSpreadsheets
Large urban	5	Surveillance IS5STD IS5HIV IS5SpreadsheetsPaper records	IIS3Statewide client management ISInventory management ISVaccine for children provider ISPractice management	VRIS4Paper recordsLocal database

1Quality, functions, interoperability, or perceptions of IS are not reported here.

2Practice management includes billing & scheduling functions

3Immunization Information System (statewide system)

4Vital Records Information System (statewide system)

5 Statewide communicable disease (and HIV/STD) surveillance systems

6Maternal Child Health Information System (statewide system)

Data Collection

Data were collected on-site in order to observe IS and data collection forms. The semi-structured, open-ended interview guide addressed job descriptions, activities, and the use and perceptions of data and IS in support of work. Research into the high prevalence of data sharing gaps [21] and existing instruments measuring IS quality [22,23] informed questionaire development. Interviews lasted approximately 45 minutes. Consent to interview personnel was first obtained from each LHD’s chief administrative officer and then from each interviewee.

Data Analysis

Analysis followed a general inductive approach [24]. Independently, JV and MI read the transcripts and employed open coding to identify tentative categories. Both team members have worked in local public health. The independently derived codes were reduced by consolidating overlapping categories and identifying higher-level themes. Discussion resolved differences and resulted in a category labels and descriptions. To validate the thematic codes, SL independently conducted closed-coding of all transcripts. A sample set of documents from each program area indicated >90% agreement between three sets of coders. Lastly, we also conducted member checking; one informant reviewed their transcript and concurred with the assigned coding.

Results

LHDs used a mix of multiple IS supported by traditional paper forms, telephone calls, and faxes to collect and share all the data necessary to complete their daily work. The IS included both state-provided and locally implemented systems as well as business process oriented IS like practice management systems (Table 1). As indicated, the IS in use were very specific. Each programmatic area used its own IS and for the communicable disease programs the IS were frequently disease-specific.

We identified 46 categories within 11 themes (see Appendix). We present the four most salient to the majority of interviewees in detail with illustrative quotes in Table 2.

Table 2

Themes and corresponding categories regarding public health information systems (IS) and technology.

Selected theme	Selected category	Specific areas & example quote
Factors affecting information system quality		Descriptions of factors, circumstances, or conditions that affect specific quality characteristics or overall quality of the data within the IS
	Information system quality - reporting / output capability	“…it’s not a report writing system in the sense that I would think it is where it generates aggregate output…when we run a report we’re basically creating another data file.” - Epidemiologist, urban LHD“We do have the opportunity to run some reports. But, to be honest with you, it's so difficult to run a report that no one does it.” – Communicable Disease, suburban LHD
	Information system quality - interoperability	“Our health department providers that use [the IIS], they're like, "Really? We have to input everything into [the IIS] and then at the end of the month we have to do it again into [Vaccine management system]?" And it would be a lot easier, yeah, if they talked to each other…” – Immunization coordinator, urban LHD
Barriers to data acquisition from others		The system level context, organizational level factors, or situations that affect the need or ability of staff to get information from other organizations or sources
	Jurisdictionally defined work	“The access we have now for neighboring counties is just that we can put a name in and we can see it's in there, but we can't necessarily see the disease or see what's going on there.” – Nurse, urban LHD“You just don't have the ability to see everything that's going on, because some things are blocked.” - Communicable Disease, suburban LHD
	Mobile populations	“Confidentiality. They don't wanna be known wherever they're going. So if they feel like they can't have the confidentiality there in [city in neighboring state], then they'll come here and be tested.” – Public health nurse, rural LHD
	Data ownership	“We need our data back, and we need it back immediately…[The SHA is] looking at it simply as data… What that means to us is much more important.” – Registrar, urban LHD
Barriers to effective data sharing (to others)		Experienced and reported difficulties, challenges or factors/situations that need to be overcome/addressed in order to provide data to others
	Reporting back	“So it seems like our staff in the unit have to pull information from [IS], put it on a separate piece of paper, and then send it to the state. So I’m not sure why we have to add that extra step when I feel like, in an ideal world, we would be able to use [the IS] to report on the information that they need since there already is a way for us to collect it.’ – Epidemiologist, urban LHD
Consequences of Data Sharing Barriers and ISQ Problems		All consequences or outcomes associated with the inability to efficiently secure desired information from other sources and of having poor data quality
	Duplication of work/re-work / inefficient work	“If you got a parent that’s not a good steward of records, they could possibly have that same child immunized about 3 or 4 times by the certain age and they don’t necessarily need all those vaccines.” – Immunization staff, rural LHD
	Workarounds	“We were having to write everything in the comment field for zoonosis.” – Public health nurse, urban LHD
	Shadow IS	“We're duplicating our reporting. We do one for in house to help us keep track, and then we use the state system.” – Communicable disease, suburban LHD

Factors Affecting IS Quality

This theme encompassed factors, circumstances, and conditions that effected specific quality characteristics or overall quality of the data within the IS. Also included in this theme are assessments or views on the quality of the IS itself in terms of the user experience, available data, and reporting features.

Reporting functionality is important for public health IS since being able to generate aggregated statistics or line listings from data is critical to case investigation and community assessments. Interviewees, however, frequently mentioned difficulty using the reporting capabilities of their IS: the system did not produce reports in a desired format; reports were too difficult to obtain; or the capability to run reports wasn’t present at all. Reporting was generally absent for vital records. The registrar in the rural LHD did not run reports, but neither did the SHA share reports with her. The complete absence of reporting functionality at the suburban LHD forced vital records staff to do manual counts off screen displays. The urban LHD could not run reports from their SHA-supplied IS either.

Instances of interoperable IS supported work existed, with immunization programs more integrated than other programmatic activities. Interoperable IS use standards to ensure the meaning and usability of data are preserved when exchanged. The rural LHD used a public health-specific practice management system that could export data into the state’s immunization IS (IIS) eliminating the need for double data entry. However, the urban and suburban’s IIS did not have true-bidirectional data sharing with other applications. Interoperability was limited to data only being able to share in one direction, restrictions on having records in one system first, or only being able to share data for children and not adults. As a result, those IIS were not truly comprehensive sources of information on immunizations or data had to still be actively managed and re-entered by staff.

Due to the high degree of centralized control over registration, the vital records IS were near to being true enterprise-wide systems, e.g. a single IS served the entire state. As a result there were no other “official” public health systems with which the IS had to be interoperable.

Barriers to Data Acquisition & Sharing

More than any other theme, the categories of data in this theme focused on the role of public health system and organizational level factors in data management. Issues fundamental to public health, like jurisdictionally defined work, measurement of populations, and data ownership, each affected how practitioners obtained or shared data. This theme was evident across all programmatic areas and LHDs.

Often practitioners knew relevant data existed elsewhere, but could not access it. For one, interviewees confirmed mobile populations fragmented client data. For example, a nurse from the rural LHD explained that a sizable percentage of her clients were actually from a large city in a neighboring state. Additionally, jurisdictional boundaries translated into restrictions on data access. This was true even for programs with shared IS. If a nurse at the urban LHD reported a case, but investigation determined the individual lived in another county, eventually her ability to view much of the detail on the case would be limited. The suburban LHD reported the same issue. For vital records, viewing and editing were also curtailed for out of jurisdiction individuals. The exception among the group was immunizations: the capability to get or view out-of-jurisdiction information was less of an issue due to shared IIS.

Issues around which agency controls data, regardless of how collected, were most pronounced at the urban LHD. For communicable diseases and vital records, staff did not have direct control over, or even direct access to the data they collected on their community. Instead they relied on extracted datasets from the SHA. Despite the lack of control, the LHD believed they owned the data.

Few practitioners reported barriers to getting data to the SHA, which was not surprising since the SHA supplied most of the main IS or the SHA was receiving paper forms. Instead, practitioners mentioned idiosyncratic work processes, the complexity of dealing with multiple departments, and narrow reasons for sharing data. For example, reportable condition staff at two LHDs mentioned having to run reports for the state when the information was already available in a shared IS. Lastly, for most locals, data sharing with the SHA was part of business processes: immunization programs for monitoring inventory and compliance; vital records for registration and issuance; and communicable disease programs were “required”. While local practitioners valued data, they generally did not recognize value in sharing data with the SHA.

Consequences of Data Sharing Barriers and IS Quality Problems

Many examples of inefficient and wasted effort could be expected from challenges with technology: double data entry, submitting extra paper copies of forms along with electronic data, multiple phone calls, and duplicated information requests. IS could even complicate the relationships with local providers if they too were forced to do double data entry.

Likewise, workarounds existed to both inputting and retrieving data. For example, communicable disease staff used comment fields to record information in the SHA provided IS. This practice was disease specific as some conditions had IS fields that corresponded to all the information captured on paper field records, but others did not. Getting data back out of systems was often difficult, because not all fields could be queried or local staff did not have access rights. To get around this challenge, communicable disease staff would call SHA employees with the sufficient access to request custom reports or for specific inquires.

Each LHD in this study used at least one shadow system, a parallel IS that only existed to provide easy access to data already stored somewhere else [25]. These additional IS ranged from spreadsheets listing cases to local relational databases designed to manage all aspects of public health reporting and analysis. The origins of the systems were primarily linked to inaccessible data, the need to retain data that could not be entered into state IS, or differences between local’s and the state’s preferences for data management and recording. Public health practitioners easily justified the use of shadow systems: their “home-grown” systems provided timelier, more complete, more accessible, more accurate, and more useful data than the “official” state repositories.

Use of Information

Across all programmatic areas, public health agencies collect data for action. Interviewees recognized the value of data to both their agencies and their constituents. Additionally, they identified applications for the data whether it was for managerial decision making, information sharing with the SHA, or for the potential to improve public health.

Unfortunately, interviews revealed IS quality, data sharing barriers, and organizational capabilities individually or combined made the turning of data into information difficult. Some of the most striking limitations on the use of information were around management decisions and strategic planning (Table 3). Lack of reporting capabilities or insufficiently detailed reports limited the ability of the LHDs to use that information for broader planning purpose. Also the lack of integration between the systems did not allow staff to “pull that information out and utilize it effectively”. Sometimes it was employee’s skillsets or competing responsibilities, but other times practitioners did not see the value in aggregated information.

Table 3

Quotes explaining the challenges to turning data into information for public health practice according to local health departments

Staff Position, LHD	Quote
Communicable disease supervisor, Suburban LHD	“I feel like there's a lot of data that comes in, but there's not a lot of data that goes back out into the community…Why are we collecting all this data if we're not informing people of what we're finding?”
Vital records staff, Suburban LHD	“The [LHD director] wants the information for statistics and sharing with city planning to see where risk areas may be…The state does have canned reports that we can request, but he wants more specific and that we cannot get it from the state or we can’t generate it ourselves without individually going through each birth certificate and pulling out the information we need.”
Epidemiologist, Urban LHD	“For the most part, the state reports, they don’t always have the data that we need at the local level. And since we can’t run reports at the local level, we do have to go back to the state to request the information.”
Epidemiologist, Urban LHD	“…People at the state have found that sometimes their own data release policy is too restrictive and there’s definitely people who work there that do realize that there is some value in releasing some aggregate data so any of the stakeholders who are interested in it could learn something from it.”
Communicable disease supervisor, Suburban LHD	“I don't know how to run reports well enough for STD yet to be able to feel comfortable pulling the data and using it for strategic planning. I would rely on our old manual paper because I know that is accurate.”
Nurse supervisor, Suburban LHD	“I’m so busy dealing with the day-to-day activities that it’s like, ‘Okay you need this report, fine I give it to you whatever,’ but truly right now I’m just kinda like overwhelmed ...”
Communicable disease staff, Rural LHD	I've heard them talk about making reports...but I really don't have the need for it right now.”
Immunization program director, Urban LHD	“The biggest challenge we have is the overall integration of the data systems. So it's not just immunizations, it's STD and communicable disease and everything that they're doing, there's health information in there, but I don't think there's anybody that really knows how to pull that information out and utilize it effectively. We have such a patchwork that there's no way, without an incredible amount labor and resource, to sort of pull those things together.”

Discussion

The process of acquiring, managing, and sharing public health data at the local level is complex. Numerous IS of varying quality and capability both support and complicate the process. Our interviews with public health practitioners revealed a need to improve data sharing efforts and activities in order to promote efficient public health practice, support decision making, and ensure confidentiality and security.

LHDs are obligated to share data on their communities with their SHA [26] and the interviewed LHDs were meeting that obligation. However, difficulties and inefficiencies permeated the entire process. These challenges are not surprising and almost natural outcomes of the complicated and multi-faceted mechanisms and processes by which LHDs collect and manage data. For example, each program needed multiple IS in order to provide services or public health activities. The use of multiple, different IS complicates work through multiple passwords, log-ins, and switching between systems [12]. If interoperability is absent in a multi-system environment, as was often the case in our observations, then double data entry and other inefficiencies result. Lastly, the continued reliance on paper as an important part of data management also contributes to inefficient work. Hybrid paper-IS data management approaches are slower and less productive than IS alone [27,28].

Several IS characteristics thwarted public health professionals efforts to turn data into information for planning and decisions making purposes. Lack of interoperability and multiple IS did not support obtaining a complete picture of the health of a community. Also important was the absence or ineffective reporting features in several IS. The ability to access information in a meaningful and easy manner is a marker of IS quality [22]. More importantly, without accessible information LHDs were limited in their ability to use evidence-based decision-making, engage in strategic planning, or undertake quality improvement efforts [3-5,29]. When LHDs do not possess sufficient internal data capabilities, they rely on the SHA [30]. In these instances, LHDs need to work with cooperatively with their SHA counterparts to identify reports that meet their local needs.

While IS quality complicated work, our finding suggest that attempts to improve data management in public health practice need to adopt a socio-technical perspective. Socio-technical theory emphasizes the interplay between IS, individuals, and their broader contextual work environments to improve IS effectiveness [31,32]. As an example of this dynamic, we documented the influence of state policies on data management and usage issues. Communicable disease and vital record staff routinely employed IS that contained information to which they felt they needed access in order to effectively do their jobs, or it was information they themselves had created. Yet, that information was unavailable solely due to policy and not due to technology. Likewise, practitioners did not always possess the skills to effectively use all of the IS capabilities. Under these types of contextual constraints or skills, problems will not be addressed by simply upgrading or buying a new IS. Given financial limitations facing many agencies nationwide, upgrades or new technologies may not be feasible anyway. Instead solutions to these problems will come from interorganizational collaboration, increasing organizational capacity, workflow redesign, and end user training.

The prevalence of shadow systems represents an area of concern. Shadow IS are a consequence of a failure, or perceived failure, of the enterprise IS to meet users’ needs [25,33]. These needs can be access to information or desired analytics and reporting capabilities [34]. In this way, shadow systems are an extreme case of a workaround; practitioners want to do their job enough that they are willing to duplicate entire systems. Shadow systems, even rudimentary spreadsheets, come at a cost: they must be created, they often require double data entry, and they must be supported and maintained. As another potential cost, they are a security and a privacy threat [35]. Older systems do not have the same security protections as newer IS and accidental disclosure of confidential public health information has happened in the past, because sensitive information was recorded on spreadsheets [36].

Finally, IS presence, quality, functionality, and governance differed by programmatic area. As a result, within each LHD, IS capabilities and experiences varied widely and staff even functioned under different access policies for patients or cases outside their jurisdiction. The categorical and disease-specific nature of public health funding [37,38] may contribute to these different experiences. For example, immunization staff had greater capabilities and policies facilitating information access, probably due to the decade long investment and national priority around childhood immunizations and IIS [39]. Current trends may address these differences in IS; specifically, public health accreditation places an emphasis on understanding data sources, technology; and community planning requires data [40]. A coupling of improved organizational awareness with flexibility around funding and investments would help address the wide variation in IS [38].

Study Limitations

We interviewed a diverse set of LHDs and explored three major program areas, but the findings may not generalize to other departments or activities. Our small sample may not be nationally representative of the experiences of practitioners, where IS and data quality issues may be common [6,7,12]. We also acknowledge that or perspective on data sharing is limited; we have no data from the SHA perspective. This would be an important avenue for future data collection as SHAs tend to have more advanced IS capabilities than LHDs, but have more partners with which to share data. Additionally, IS is a critical mechanism to link the efforts of public health and the healthcare system [41], but our interviews did not fully investigate that area. Given the current national emphasis on health information exchanges and electronic medical records [42], we acknowledge that the results reflect a current reality that might be rapidly changing in many LHDs.

Conclusions

Local public health involves the use of multiple IS supported by paper records. Current approaches to data management and sharing do not always support efficient public health practice nor allow data to be used for organizational and community decision making. Some of these challenges can be addressed through SHAs cooperatively working with LHDs in the state to define standard work processes and to establish IS governance that supports local practice.

Themes and codes with definitions used identified from content analysis of interviews with local public health practitioners.

Theme	Code	Definition
Barriers to data acquisition from others		The system level context, organizational level factors, or situations that affect the need or ability of staff to get information from other organizations or sources
	Jurisdictionally defined work	Differing roles and IS responsibilities based on political areas/jurisdictions or geographic areas of program implementation/oversight, as it pertains to subsequent acquisition of data
	Mobile populations	Citizens/patients receive services at various locations which fall under different jurisdictions with different IS/ forms/ policies, as explanation of why data acquisition is difficult
	Data ownership	Description of which organization(s) or departments control/owns which data elements or overall data that are needed, as it pertains to subsequent acquisition of data
	Data access control	Nontechnical aspects and policies of departments that determine who can access /use what information in existing IS; blocked access to needed data elements
Barriers to effective data sharing (to others)		Experienced and reported difficulties, challenges or factors/situations that need to be overcome/addressed in order to provide data to others
	Multiple data partners at state	Having to deal with different state offices / agencies / departments for data related to a given health topic
	More than 1 IS to do job	Job or single task requires the access / use of more than 1 information system
	Task technology fit	Issues regarding the match or appropriateness of the design of the IS to public health work, including fragmentation of the IS across agencies/departments
	Organizational capabilities	Skills within the organization /department (analytic, technical) to be able to use information, as antecedent to sharing data or reports
	Reporting back	Lack of information flow back from other organizations and departments with whom data had been shared (ie, reports) regarding use or quality of those data; no feedback loop
Organizational sharing partner		Attribute code to clarify / describe who data sharing is occurring with.
	Other LHD	Other local health departments in other jurisdictions.
	State health agency – general	The state health agency in general – not specific to any unit or department with the agency
	State counterpart department	The counterpart department with the state health agency (e.g. immunizations, communicable disease, vital records)
	Providers	Any healthcare organization, provider or physicians
	Other	All other organizations (funeral homes, charities, social services, etc)
Factors affecting ISQ		Descriptions of factors, circumstances, or conditions that affect specific quality characteristics or overall quality of the data within the IS
	Sources of Error	Explanations of how errors are introduced into the data, general comments regarding sources or extent of errors
	Data logistics/Work processes	Descriptions of the work process of collecting & reporting information and the ways those work processes are related to specific or overall data quality
	Use paper for job	Instances where paper is required or used in parallel with IS (both forms & as a paper-based record keeping) in order to have complete data (not shadow system)
	ISQ timeliness	Issues affecting or perceptions of the timeliness of the data in the IS
	ISQ missing Info	The type or extent of information that is missing in the IS, either specific data elements or entire records
	ISQ accuracy	The type or extent of information inaccuracies, such as wrong values or unbelievable information
	ISQ accessibility	Technical and software factors related to the availability and retrieval of information from the IS; user friendliness of the IS interface
	Security and confidentiality	Issues related to assuring the security and confidentiality of the data as they effect data quality (ie, ability to edit and correct data), irrespective of data ownership
	ISQ multiple data sources	Factors related to the quality of the information due to multiple users (ie data managers, data entry personnel, providers) or multiple sources of the information
	ISQ interoperability	Factors related to the ability of the IS to export/import data from other information and computer systems
	ISQ reporting / output capability	Ability to manipulate the data or generate output/reports using the existing software
	ISQ inclusion rules	What makes individuals eligible to have their data included in the IS
Consequences of Data Sharing Barriers and ISQ Problems		All consequences or outcomes associated with the inability to efficiently secure desired information from other sources and of having poor data quality
	Duplication of work/re-work / inefficient work	Repeat of work or use of inefficient work practices (ie calling to get missing data) stemming from having poor IS quality or data logistic procedures
	Workarounds	Additional work processes and communication efforts developed and used to overcome /get around /avoid barriers encountered in data availability or use
	Shadow IS	Creation and use of additional IS or duplicate IS, databases, or repositories, due to accessibility or functionality issues, in order to store and use information already available in other system, such that staff do not work from a single IS
	Effects decision making	Limits strategic planning, community planning, environmental scanning, or community assessment resulting from incomplete data
	Effects health problems	Negative effects for health of individuals or populations resulting from IS problems
Hardware Considerations		Descriptions of concerns, issues, or experiences specifically related to computer hardware and its maintenance
	Hardware & data backups	Comments regarding the characteristics of the hardware which affects its usefulness (ie interface of parts), and of the degree of capability to maintain backups
	System stability	Comments regarding the reliability of the hardware in terms of having an overall stable computer system (ie, not crash)
Data Quality focused Solutions		Descriptions of anticipated or actual ways identified to correct or overcome the known problems with the actual data elements
	Identify & correct error	Actions to pin-point the incorrect data element or record, and the associated actions to correct that specific error in the data
	Technical support	Support options available to help with issues / correct mistakes or provide analytics
Sharing focused Solutions		Descriptions of anticipated, potential or actual ways identified to correct or overcome the known information sharing problems
	Regionalization	Changing of jurisdictional limitations to focus on larger community areas as means to improve data sharing
Benefits of IS		Descriptions of perceived or actual advantages to individuals and populations, and organizations from having an IS
	Customer benefits	Benefits seen by customers, citizens, society from having accurate, timely data
Use of information		Descriptions and explanations of how public health practitioners apply (or fail to) information
	Useful information	Comments on the overall usefulness/not of the data in IS
	Management	Executive, administrative or managerial uses of information to help the work and operation of the LHD
	Community partners	Sharing of information and reports with the community
	Practitioners	How individual practitioners apply information to public health activities
	Other government partners	Sharing of information with other government partners
	Required state reports	Use of information in required reports to the state
	Required sharing	Instances of mandatory or obligatory reporting to other agencies
IS Uers’ Views		Descriptions of personal views, opinions and perspectives on the current and future of the IS that individual is working with
	Personal responses to IS	Emotions (positive and negative) triggered by working with the IS
	Meaning of information & data	Distinctions made or differences mentioned between data and information
	Full vision	Re-thinking about how IT/IS should support public health and what changes should occur to new systems