Successful Public Health Information System Database Integration Projects: A Qualitative Study.

OBJECTIVE: To explore the most important public health information system database integration project success factors to include: technological, organizational, project-specific, or external.
METHODS: This study involved a cross-case design. Cases were identified through literal replication logic and screened through a survey and review of available literature. Study participants were interviewed through hour-long sessions steered by a semi-structured guide. Survey responses, interview transcripts and available documents were coded and analyzed deductively, and matrices were developed to illustrate relationships.
RESULTS: Leadership among the project's participants is the most important integration project success factor. This leadership factor manifests in the following ways. Executive sponsors champion the initiative. Informaticians facilitate communication and system requirement collection. Program directors contribute substantive energy to the project and remove obstacles. Some other factors also contribute to project success. For example, strong Financial Management and Support promotes project initiation. Technological aspects impact the final product's quality. Utilizing formal project management techniques, particularly the Agile software development methodology, contributes to successful project resolution by ensuring daily operational effectiveness. DISCUSSION: The principal finding illustrates important contributions by project leaders, transcending those of the executive sponsor. Other participants, notably informaticians and program directors, substantially contribute to the project's success. Additionally, the Agile software development methodology is emerging as a successful approach to project management for these and related projects.
CONCLUSION: Investing in the leadership and project management skills of database integration project participants could improve the success of future projects. State health department staff considering these projects should carefully select project participants and train them accordingly.

Introduction

Public Health Information Systems (PHIS) are foundational components of public health infrastructure, providing how health departments collect and maintain data for public health practice [1]. These data support population health services such as controlling outbreaks or designing health promotion programs targeting teen smokers. State governments often establish PHIS through the state health department, and the systems primarily serve state and local health department data needs [2].

The use of information technology to develop databases is a critical aspect of PHIS. These databases store public health data, and advances in information technology have improved the ability to develop databases that suit specific program requirements [3]. Specialized state public health databases have propagated because of this technical development ease and categorical funding incentives. Computing advancements have also readily allowed for the integration of separate databases [4].

Database integration often entails the development of a common database for the organization that consolidates operational data from multiple sources [5]. When individual databases are integrated, they collectively create more complete records by piecing together different data elements from different sources. Access to complete records can improve coordination of public health activities and reduce costs [6]. The Joint Council of Governmental Public Health Agencies suggest that 1) health departments must integrate databases and 2) these integrated databases must meet information needs at the service delivery level [7].

Previous child health information system (CHIS) database integration activities illustrate the following: the development of a specific PHIS database integration business case, actions required to successfully execute the project, and prior integration project evaluation efforts [8]. A large measles outbreak in the late 1980s prompted public health and healthcare leaders to evaluate data collection and usage techniques, which led to the initiation of CHIS database integration efforts. A workgroup identified programs such as immunizations and vital registration as a suitable starting point for the integration projects [9]. Evaluation activities included documenting and studying the critical success factors for these integration projects. Findings from CHIS integration studies informed research in related areas [10].

Customized program-specific databases have proliferated but often they have not integrated with other databases throughout the health department. Many public health program managers have established databases without considering broader database integration. These databases meet the individual program’s data needs, but do not address enterprise information management needs across the organization [11]. Silo public health databases result in inefficiencies, such as poor disease control and outbreak response coordination; incomplete service delivery at the local level; and underperforming population health protection measures during public health emergencies [12]. While leaders integrated and evaluated some CHIS databases, few other successful PHIS database integration initiatives have been studied. It is not known which PHIS database integration project factors are most important and how they impact successful public health database integration projects.

This study explored factors that most contribute to successful intra-state public health information system database integration projects. Technological factors, such as the quality of data within the originating data systems, impact the functionality of the integrated database. Organizational factors, such as meaningfully engaged executive leadership and strategic plans, impact the agency’s readiness and commitment to the initiative. Project-specific factors, such as effective governance and formal project management techniques, impact the day-to-day administration of the initiative. Other factors outside the agency’s direct control, such as data privacy laws or the actions of external stakeholders can also influence the project’s success. The Health Resources & Services Administration’s Sourcebook lists the nine non-technical integration project elements (factors) [13]. Table 1 includes this list in addition to the technical factors. All factors have been grouped into logical Domains for this study based upon prior research [14-21].

Table 1

Integration Project Success Factors, Grouped

ORGANIZATIONAL DOMAIN

Leadership

The project has an executive sponsor, a high level official who advocates for the project, and a champion, someone who is willing to devote a significant effort to see the project succeed.

Organizational and Technical Strategy

The project has a strategy that takes into consideration local issues such as funding, the political environment, organizational structure, the strengths of the organization, and stakeholder beliefs and values. The selected technical integration approach accounts for internal data governance and data sharing needs, which must conform to state and federal laws and agreements made with stakeholders.

PROJECT ORIENTED DOMAIN

Project Governance

The project is guided by a steering committee representing all key stakeholders. The steering committee develops the integration strategy, based on clearly defined business processes.

Project Management

The project has formalized management strategies and project management methodologies designed to assure consistent communications, accountability, and resource constraints.

Technical Support and Coordination

Technical information systems support and coordination is organized centrally to assure consistent support and a robust infrastructure capable of maintaining and complying with standards. A business analyst supports implementation.

Financial Support and Management

The project is adequately funded and has multiple funding sources.

Evaluation

The project has some form of qualitative and/or quantitative monitoring or evaluation that is performed regularly.

EXTERNAL DOMAIN

Stakeholder Involvement

Frequent communication with stakeholders and involvement of stakeholders in the integration project throughout the life cycle of the project contributes to its success and credibility.

Policy Support

Rules, regulations, legislation, and policy advisory or policymaking bodies are supportive or at least neutral to the integration of health information systems. Executive sponsors educate policymakers about sensitive issues to garner their support.

TECHNICAL DOMAIN

Source Systems

Databases contain quality program-specific data to be contributed to the database integration project.

Development Technology

Project managers select a particular technology to be utilized for the integration project including architecture, hardware, database software, data integration engines, user interface, etc.

This manuscript will also describe how Agile software development facilitated the daily project management for many of the PHIS database integration initiatives illustrated in this study. The use of the Agile software development methodology in public health practice is poorly understood. Agile software development, or simply “Agile”, is emerging as a popular software development project management alternative to more traditional approaches such as the widely-used waterfall methodology. The Waterfall model entails a prescriptive stage-oriented software development process characterized by exhaustive initial requirement collection and design phases [22]. Agile is considered a “lightweight” method for developing software, with principles that focus on intensive collaboration and rapid software iteration versus extensive up-front system requirement documentation and highly-regimented planning [23].

Many technology companies utilize Agile to rapidly iterate software products and gain a competitive advantage. Organizations have utilized Agile to create software for healthcare applications [24], and others have modified aspects of the organizational culture by adopting Agile practices for managing other types of projects [25]. Researchers have studied their experiences in utilizing Agile to create and maintain biomedical software, and found the Agile approach to be a good fit for these projects [26]. Following the failed rollout of HealthCare.gov, some departments of the United States federal government immersed themselves in Agile methodology with some success [27]. Implementing the Agile methodology does not come without its risks for failure, but its success factors have been studied [28]. The role of Agile in the PHIS database integration projects identified in this study will be illustrated further.

Methods

A cross-case study design was used for this research. The state health department is the unit of observation, and the database integration projects are the unit of study.

Case Selection

The researcher identified cases through literal replication logic, seeking successful state health department intra-state public health information system database integration projects [29]. A three-phase screening procedure was utilized, entailing 1) deploying a survey; 2) reviewing successful PHIS database integration projects; and 3) reputational case selection. The survey targeted state health department informatics staff as members of the Association of State and Territorial Health Officials (ASTHO) Informatics Directors Peer Network (IDPN). The survey was administered to IDPN members, and the first phase of screening reduced the candidate list to cases that most closely fit the literal replication design. Seven participants responded to the survey, and three met the criteria for additional follow-up.

Next, the researcher identified and screened cases based on criteria specified through a literature review of frequently documented, successfully completed PHIS database integration projects. Peer reviewed literature, books, and federal agencies have documented these projects, and this literature illustrates best practices and exemplars. In particular, CHIS database integration research and studies evaluating the Environmental Public Health Tracking activities provide substantive insight into successful PHIS database integration initiatives [30]. The researcher contacted representatives from state health departments presented in these research bodies for inclusion as participants in this study. Three cases were selected through this literature review process. Finally, one case was identified through reputational case selection referral by the Public Health Informatics Institute. Through these processes, the researcher retained seven cases that best fit the literal replication design of successful PHIS database integration projects. The study protocol was reviewed and approved by the University of Illinois – Chicago Institutional Review Board.

Survey and interview questions

The researcher adapted questions from a previous CHIS database integration study [31]. The survey questionnaire addressed case demographics and questions that pertain to the technical and project planning domains. The interview guide was piloted with four informaticians from state health departments, and the final version was organized into the Domain groupings from Table 1. The interview guide asked participants about the agency’s informatics projects and the impact of each Domain’s factors on the integration project’s success.

Procedure

The researcher conducted and recorded approximately one-hour semi-structured interviews with state health department informatics directors along with referral follow-up interviews of program directors, bureau chiefs, system administrators, and technical staff. The researcher interviewed twenty-five participants through nineteen interviews (some interviews included two participants) from April to September of 2016. The survey responses were then paired with associated interview transcripts. Finally, the researcher obtained from participants and websites copies of pertinent project documentation for review: strategic information technology plans and operational plans for the database integration projects; agency budgets; project meeting minutes; project charters; progress reports; policy documents; and protocols.

Analyses

The analysis included within-case analysis followed by cross-case analysis. The within-case analysis entailed comparing and summarizing survey and interview responses, then contrasting these summaries with the document content analyses. Once this was performed for each case, the cases were compared.

Data were analyzed using a priori theory-based codes with deductive coding: assigning labels to data to summarize the basic topic of a qualitative data passage. The coding began with an initial list of codes based upon pre-existing theory, largely from the CHIS database integration studies [32]. Additional codes emerged inductively. All data analyses were performed using ATLAS.ti to code and compare thematic survey responses, interview transcripts and the document contents [33]. A common codebook was established serving as the base for all analyses. Data display matrices were created to illustrate systemic relationships and the within-case and cross-case synthesis.

Results

Study participants discussed state PHIS database integration projects and the factors that contributed to the projects’ success. Supporting documentation substantiated participants’ claims. Chart 1 illustrates quote frequencies from the survey responses, interview transcripts, and codes from the document review, and it provides a basis for drawing initial conclusions.

Chart 1

- Integration Factor Quote Frequency

These counts principally illustrate how much the participants spoke about any of the integration project factors, as specified through the coding process. The technological aspects of the integration projects are discussed most frequently, followed by leadership aspects or cross-cutting departmental projects more generally.

Leadership emerges as the most important factor

After obtaining information about how the factors contribute to the project’s success, participants were asked which factors were the most important and why. These factors are illustrated in Table 2, which organizes each interview grouped by case, with columns illustrating the most important project integration factors as reported by the interview participants and an overall conclusion from the researcher’s perspective. Leadership aspects of the projects predominate in importance. In particular, participants highlighted the leadership roles of executive sponsors, program directors, and informaticians. Financial Support and Management; Project Management; and the project’s Technology are three other factors that regularly surfaced as important project success factors.

Table 2

Reported most important project success factor and explanation

Participant by State	Most important reported factor(s)	Researcher's explanation
State BParticipant 4	-Financial Support-Organizational Strategy-Technical Support & Coordination	Organizational alignment and accreditation set the stage. Executive staff serve as project champions. Informatics business analysts make a difference. Demonstrating value secures flexible funding.
Participant 5	-Financial Support-Informatics Leader	Dedicated funding is crucial. Informaticists bridge communication gaps.
Participants 6 & 7	-Financial Support-Technical Support & Coordination	Project completely stalled when the funding temporarily vanished. Well-defined system requirements propel the project.
Participant 8	-Leadership, executive*-Financial Support	Executive champions play a critical role. Funding is crucial, and can be frustrating.
Participant 9	-Leadership, executive*-Financial Support	Executive champions and project funding are crucial.
State FParticipants 19 & 20	-Leadership, program*-Informatician leader	Team dynamics and personalities make or break the project. Program-level leadership, not executive support, makes the most difference.
Participants 21 & 22	-Leadership, program*-Informatician Leader-Technology-Technical Support & Coordination	Interaction between the tech team and business analyst/informatician is critical. Division-level (not executive) leadership facilitates success. A competent and capable information technology team is key.
State AParticipant 1	-Organizational Strategy-Informatician Leader-Policy Development	Organizational changes linked to strategic planning can have a big impact. Informatician leaders have an enterprise approach. Effective policy facilitates technical decisions.
Participants 2 & 3	-Technology-Project Management	Source data matters. Dedicate a tech person to the project. Strong project management includes subject matter experts.
State EParticipant 16	-Leadership, executive*-Informatician Leader	Engaged executive leadership provide vision and support, and can facilitate practical changes, such as the shift to Agile project management. Informatics staff lead the projects.
Participant 17	-Project Management-Leadership, program*	Moving to Agile from Waterfall profoundly impacted the project's success and augmented team synergy. Program directors provide substantive leadership.
Participant 18	-Leadership, program*-Financial Support	Sustained program director leadership was crucial. Agency timing was right--the will emerged. Federal grants were critical.
State GParticipant 23	-Leadership, program*	Relationships are important. Program director leadership remove obstacles and ensure team synergy.
Participant 24	-Leadership, program*-Informatician leader	Teamwork is most critical. The involved programs have the same program director and they frequently collaborate. Lead informatician is instrumental in making it a success.
State DParticipants 14 & 15	-Financial Support-Leadership, executive and program*	Federal funding has been critical. The first phase of the projects directly involves senior leadership, whereas latter phases require program leaders to step up.
State CParticipant 10	-Financial Support-Informatics leader-Leadership, executive*	Federal funding for a related initiative was leveraged for this project. Informatician and IT tenacity are critical. Senior-level support and interest are required.
Participant 11	-Informatics leader	Team dynamics achieve the outcomes. Informatics capacity must be carefully maintained or it can erode.
Participants 12 & 13	-Technology-Leadership, program*	Standards makes much of the work possible. Program directors facilitate project success. Funding plays an important prioritization role. Data sharing agreements are necessary.

*Illustrates the distinction between executive-level and program-level leadership.

Further exploration into the leadership dimension illustrated important nuances. Participants agreed upon the contributions of the informatics staff involved in the initiatives. Informaticians seemed to facilitate the collection of system requirements, translated and communicated needs across project participants and to project sponsors, and developed collaborative team dynamics. However, the contributions of executive leadership and that of program directors were less universally-acknowledged. Some participants attributed project success to the involvement of meaningfully-engaged executive sponsors, whereas others suggested success was due to the regular involvement of program directors directly impacted by the project.

Agile emerges as a promising project management technique

Project management was indicated by many participants as an important success factor for these initiatives. Table 3 illustrates the project management technique used for each case’s project and a summary of the technique’s impact on the project’s success. All but one case referenced Agile project management in some way, and the participants appeared to express favorable opinions about the role the Agile project management methodology played in the project’s success.

Table 3

Project management techniques by case

Case	Project Management Methodology	Project Management impact on project’s success
State A	Agile with Scrum specifically	Scrum techniques facilitate project management. Agency recently moved to Agile model.“In terms of the meetings and stuff go, we are using the Scrum process here, an Agile Scrum process for development, which was also a big change. We used to use Waterfall… But it’s proven that it’s working pretty well, since we switched a couple of years ago.”
State B	None, although vendor possibly utilized Agile	Minimal impact from participants’ perspective.“I think they used the Agile method with short sprints.”
State C	Agile	Agile methodology referenced by one participant but not by others.“We do Agile development. So pretty standardized as far as project management, planning and the reporting is concerned.”
State D	Agile	Regular, sustained activities move the project forward.“[The NEDSS] uses the agile development approach. All the local users’ representatives really committed a lot of time to do it.”
State E	Agile	Moved to Agile from Waterfall methodology and this change has had a substantive impact, including leading other areas of the agency to adopt the methodology.“We also have moved from a Waterfall method for project management to an Agile methodology. It has made all of the difference in the world; I cannot tell you what a difference it has made. It’s been incredible.”“I think that if we hadn’t had Agile, we still wouldn't have a system up. We would have trashed the build and still wouldn’t have something.”
State F	Waterfall for most projects but Agile for one	Consider Agile to be the better method but not used consistently across the organization.“And Agile to me was superior and definitely what we should implement here.”
State G	None	Minimal impact from participants’ perspective.

The participants described that by utilizing Agile, the program staff, informatics personnel, technological developers, and others involved in the project closely collaborated in the development of the integrated database, producing a better product and overall experience than that through traditional software development techniques. One case’s participants in particular, State E, stated that the shift to Agile from the Waterfall methodology introduced a profound cultural shift within the agency that transcended PHIS database integration and general information technology project management. Other areas of the agency began applying Agile methodology to other projects based upon the success identified in its use with the PHIS database integration projects. Participant 16 described this profound shift:

“Since that time we don’t do anything but Agile. What’s really cool is some of our business side—our service areas—want to start using Agile with their staff, because it holds people accountable. You have to stand up and say ‘This is what I did yesterday, and this is what I’m going to do today.’ Everybody gets to hear it; everybody has to be accountable. It builds that team that you just don’t always see with things like that. It was a profound difference, I’ll tell ya.”

This shift to Agile methodology had substantively changed the project management experience for some of the study’s participants. Participant 17 from State E suggested that the utilization of the Agile methodology was the most important success factor for the project:

“I definitely think it’s the Agile process in and of itself. It helped the project move forward. Even when we had a roadblock it’s not like everything just stopped…It created this wonderful team atmosphere where everybody knows we’re working for this same end goal.“

“Switching to Agile made a huge difference, and I would recommend it for any process.”

“So it’s a very interactive, engaged process. It’s incredible, I’ll tell ya. I’ve been amazed at the differences—-the speed at which a project gets done. All of that front-end time is lost.”

Agile clearly changed the way the health department conducted business, and positively impacted many of the integration projects described in this study.

Consistency across cases

The case summary table (Table 4) illustrates cross-case comparisons and distinct features of each. The participants’ remarks from each case seem to consistently reflect across the spectrum of cases. Funding is cited as a project catalyst, and Leadership involvement across multiple levels of the organization ensures project success in various ways. Technological factors such as the quality and structure of source data, ensuring standards are employed, and maintaining data warehousing expertise impact the development of the integrated database. Effective project management facilitates project success, and Agile is regularly referenced as a useful method.

Table 4

Case Summary Table

State	Summary
State A	Strategic planning and policy development set the project context. The Informatician plays a critical role by fully engaging team members. The quality of the source data impacts development progress.
State B	Executive leaders align resources and seed funding. Informaticians collect thorough business requirements. Prior strategic planning that addressed information management principles seems to have had a lasting positive effect.
State C	Leadership by the program staff and informatician ensure functional team dynamics. Technological standards facilitate other programs’ integration efforts. Executive support and interest bolster project activities. Funding is critical.
State D	Executive leadership set the project vision and initial activities, and strong program and bureau leaders are required for project sustainment. Funding is essential.
State E	Program and executive-level leadership both impact the project. An agency-wide shift to Agile project management changed the organizational culture and facilitated success. Informatics staff lead these initiatives. Funding was crucial.
State F	Program-level leadership and informatics leaders promote healthy relationships and team dynamics. Information Technology team member permanence ensures continuity.
State G	Program directors and informaticians ensure success by fostering functional team dynamics and relationships.

However, important differences surface when participants describe the contributions of the executive leaders compared with those of program directors. As alluded to in Table 2, some cases evenly highlight the contributions of both groups, whereas other cases are characterized by substantive involvement of either executive leaders or program directors, but not both.

Discussion

Principal Findings

The study’s principal findings illustrate the complex involvement in PHIS database integration project leadership beyond the role of the executive leaders. Much is known about how executive leaders contribute to project success through their sponsorship and support. This study suggests that other project participants, notably informaticians and program directors, substantially contribute to the project’s success. While executive involvement might be critical for initiating the project, program directors ensure project staff remain engaged, and informaticians provide a crucial role in facilitating project conversations across diverse participants.

Additionally, the Agile software development methodology is emerging as a successful approach to project management for these and related projects. Some participants claim adopting this approach introduced a dramatic shift in how the integration projects progressed, and one suggested this was the main reason that site’s project succeeded. Agile improves project accountability and team member involvement and interaction, while speeding the deployment of useful software.

Implications

This study has three primary implications. First, developing the leadership skills of informaticians, relevant program directors, and executive leaders may promote the success of these and related initiatives. Since these projects require informatics savviness, these individuals may benefit from informatics training more generally, and PHIS database integration training specifically. Secondly, project financing challenges are not new to public health departments, and this aspect seems to impact PHIS database integration project success, especially the launching of these initiatives. Federal programs have funded these efforts in the past, and future funding could facilitate their initiation. Finally, employing formal project management techniques might ensure the project runs smoothly. Investing in Agile methodology training and enabling its use could be an effective approach to ensuring the project is properly managed.

Limitations

This study has three principal limitations. First, cases purposefully recruited represent an exemplary and small subset of all state public health departments. Therefore, the study’s results should not be interpreted as representative of all state health departments. Secondly, the data are based on survey responses, interviews, and a document review. Participant responses may be affected by subjectivity, and undiscovered documentation may suggest alternative conclusions. The data had not been triangulated with onsite visits and additional observations to corroborate findings. Thirdly, a single researcher performed the data collection, coding, and analysis. Inclusion of another researcher could confirm codes and findings. Despite these limitations, the study’s findings provide useful insight into integration project success. More research in this area is needed to further understand this topic.

Conclusion

This study improves the understanding of the most important public health information system database integration project success factors. Public health database integration needs persist, and stakeholders may use these findings to improve the likelihood of future project success.