Strengthening Partnerships along the Informatics Innovation Stages and Spaces: Research and Practice Collaboration in Utah.

Collaborate, translate, and impact are key concepts describing the roles and purposes of the research Centers of Excellence (COE) in Public Health Informatics (PHI). Rocky Mountain COE integrated these concepts into a framework of PHI Innovation Space and Stage to guide their collaboration between the University of Utah, Intermountain Healthcare, and Utah Department of Health. Seven research projects are introduced that illustrate the framework and demonstrate how to effectively manage multiple innovations among multiple organizations over a five-year period. A COE is more than an aggregation of distinct research projects over a short time period. The people, partnership, shared vision, and mutual understanding and appreciation developed over a long period of time form the core and foundation for ongoing collaborative innovations and its successes.

Introduction

Public health informatics (PHI) is an action-oriented science and innovation-driven practice. Partnership between academic informatics researchers and public health practitioners is crucial for successful translations of informatics research findings into practice. Building sustainable partnerships over various innovation journeys and efficiently translating a product from a research laboratory into public health operations are challenges for the academic Centers of Excellence (COE) in Public Health Informatics. In this manuscript, we describe the Rocky Mountain Center of Excellence in Public Health Informatics (RMC) and put it forward as an illustrative example of a framework for successful innovation partnership between public health and academia in Utah.

We built the RMC upon ongoing collaborations from three related domains: Informatics research, epidemiology and population sciences research, and public health practice (see Figure 1). The research institutions include the University of Utah Departments of Medicine, Pediatrics, and Biomedical Informatics, and Intermountain Healthcare. The practice organizations include Utah Department of Health, Salt Lake Valley Health Department, Davis County Health Department, Denver Health, and Intermountain Healthcare. Regardless of a member’s background, all the collaborators had a shared vision of the RMC as a “Center of Excellence to rapidly translate public health informatics research results into practice, to broaden collaboration between innovative researchers and practitioners, and significantly impact and improve epidemiology and surveillance.” The RMC’s innovation priorities include informatics tools to support disease surveillance and investigation (especially food safety, pandemic influenza), public health case reporting, and secured communication.

Figure 1:

Interactive domains of public health informatics

Shared vision does not naturally lead to smooth partnership among different disciplinary professionals. Thomas J. Allen pointed out that to treat both professions (engineers and scientists) as one and then to search for consistencies in behavior and outlook is almost certain to produce error and confusion of results (1).An epidemiologist’s priority, in general, is to conduct disease surveillance, investigate outbreaks, and operate a public health system under the public health legal authority. A researcher’s priority is to conduct studies and disseminate novel discoveries directed by funding sources with time limits. Aside from the disciplinary factors, collaborative informatics research among academic and public health parties often involves members of one party performing in the workspace of the other. Workspace crossover may lead to tension among partners. Furthermore, content of partnerships and interactions across workspace at different innovation stages are interdependent but not consequential as well (1).

Framework for Public Health Informatics Innovations’ Spaces and Stages (PHI-ISS)

In order to translate our innovation experiences into a framework to better coordinate the research projects across research and practice spaces and project lifecycles, we used inductive reasoning methods and theorized our own experiences during the past five years. Specifically, we first collected information from one project such as project deliverables, tasks, timelines, responsible parties and locations. We analyzed the information within the context of our own recalls and reflections on the partnership situations for each of project deliverables and tasks. For example, the concept of the “innovation space” was first used by a public health investigator during a heated discussion on how to balance resource commitments between research project timeline and public health practice priorities. When we summarized our research activities and evidences of partnership development into a structured framework, the concepts of innovation “space” and “stage” emerged.

Public health informatics is a relative new sub-field of the health informatics domain, which also includes clinical informatics and has recently seen an explosion of interest in funding and conducting translational research. The paradigm of clinical translational research has a four-phase conceptual model describing clinical research progressing “from bench to bedside.” We adapted and applied this phase concept to public health informatics research arguing that the corollary is to transition public health informatics research and innovation “from campus to field.”

Clinical and translational research has four distinct but interdependent phases (T1–T4):

T1: research seeks to move a basic discovery into a candidate health application

T2: research assesses the value of T1 application for health practice leading to the development of evidence-based guidelines

T3: research attempts to move evidence-based guidelines into health practice, through delivery, dissemination, and diffusion research.

T4: research seeks to evaluate the “real world” health outcomes of a T1 application in practice(2).

Out Public Health Informatics Innovation Stage and Space (PHI-ISS) also include four stages. We also add a new dimension of “space” in each stage as follows: Figure 2 outlines the four innovation stages where research space is highlighted in light color and public health space in dark color. The direction and color of an arrow indicate an action initiator, role impact and location of workspace. For example, at the Initiate Stage I, public health is needed to identify what the problem, issue or desired outcome is. Informatics researchers determines the “how.” Researchers mainly work on campus and make impact on public health’s participation in the Collaborate Stage II. Public health makes active input on research designs in Stage II where workspace crossover begins. At the Translate Stage III, researchers deliver prototypes, pilot products, and modify public health informatics infrastructure. When a research product is implemented in a public health practice, research impact will go beyond the original practicing collaborators being diffused widely. Over the four stages, research activities highlighted in light color gradually penatrate into public health practice indicated by the incressed dark-color areas.

Figure 2.

Innovation Stage and Space for public health informatics

ISS1 - Research initiates to move new discoveries into candidate public health applications with practitioners’ input (Space=University mainly. Finished during the grant writing period.)

ISS2 - Research and practice collaborate to assess the value of ISS1 discovery for public health practice leading to evidence-guided innovation (Space=University and public health)

ISS3 – Research translates and practice implements fully-tested ISS2 innovation into public health practice through researchers’ delivery, dissemination, and diffusion efforts (Space=Public health and university)

ISS4 - Research evaluates the “real world” impact of public health outcomes of ISS3 implementation in practice (Space=University) (3).

During the past five years, the RMC has performed research and development resulting in projects that, as of 2011, are at various points along the spectrum of Public Health Informatics Innovation Stage and Space. We now provide several examples below to illustrate the PHI-ISS framework.

Epinome: an example of transition through the continuum of Innovation Stage and Space

We describe the Innovation Stages and Space framework using our work with a new software innovation called Epinome(4, 5).Epinome is a user centric visual analytics system that empowers users to visualize, explore and analyze public health data. Epinome features a dynamic environment that seamlessly evolves and adapts to user tasks and focus change. Translation of Epinome into a public heath setting can significantly enhance the analytical capacity and usability of the reportable disease data collected using Utah’s surveillance computer system (UT-NEDSS implemented with TriSano software).

ISS1 – Initiate to move new discoveries into candidate public health applications (Space=University): 2007–2009

Started in 2007, the University of Utah’s researchers conducted interviews with state and local epidemiologists on the topic of pertussis outbreak investigation, and collected input from experts in cognitive psychology, infectious disease epidemiology, and mathematical modeling of infectious disease and visual analytics. Based on the identified public health needs, COE investigators developed a software prototype of the Epinome novel technology. The researchers then demonstrated the prototype of Epinome to public health practitioners using large scale high-fidelity simulations of pertussis outbreaks. In 2009, the researchers and public health personnel jointly wrote the RMC grant application for further development of Epinome system and seek to generalize and adapt the pertussis investigation model to food-borne illness and influenza.

During this initial stage, activities mainly occurred on campus; public health had minimal participation and less commitment to the research project. The minimal interactions between the teams did not lead to a noticeable tension within the partnership.

We applied the PHI-ISS framework to analyze collaborative activities and identified 15 deliverables from the Epinome research team as candidates for public health adaptation and implementation. Of the 15 deliverables of Epinome components, nine were in the ISS2 stage, four in the ISS3 stage, and two in the ISS4.

ISS2 – Collaborate to assess the value leading to evidence-guided innovation (Space=University and public health):2009 – 2011

At the ISS2 stage, researchers and practitioners jointly assessed the practical and potential added value of Epinome. We conducted contextual inquiries and observations of public health practice, processes, procedures, and policy to elucidate design objectives, restrictions and requirements that may be imposed at a public health practical setting. Public health was actively involved in evidence collection in this stage. The researchers conducted multiple on-site demonstrations/explorations with different groups of potential public health users of Epinome to assure that Epinome would meet practitioners’ needs. This effort has led to the reengineering of Epinome’s functionality and IT infrastructure. Researchers worked with two groups of practitioners, namely epidemiologists and information technologists, on the nine deliverables for this phase of the project: The ISS2 deliverables were workspace crossovers without significant tensions among partners as the efforts focused on information collection and knowledge exchange and were less invasive in other party’s business processes. This stage requires increased resource commitment by the public health side (e.g., face to face meeting time) to provide additional information, artifacts and consultation to researchers. At time, the research agenda and time commitments had to compete with public health operational priorities.

State epidemiologists, informaticists, and campus researchers conducted contextual inquiries on (a) food-borne disease investigation workflow at state and local health departments and (b) unmet needs from the users of the Indicator-based Information System for Public Health (IBIS-PH)

Conducted an “Affinity Walk” to organize and categorize similar user needs collected through contextual inquiries (an affinity diagram)

Developed use cases based on affinity diagrams

Develop Epinome’s ontology for processing the UT-NEDSS data

Integrate Epinome with the state of Utah’s GIS service

Develop Web services for Epinome

Ensure interoperability of Epinome with the UDOH IT environment (Authentication, application/database servers)

Mirror the UDOH implementation environment (UT-NEDSS functionality, export schema, Apache/Spring server respond to user’s requests) at the University of Utah’s IT laboratory

Create visualizations of diagnostic Pulsed Field Gel Electrophoresis (PFGE) patterns (an alphanumeric result) that assist epidemiologists in understanding both the pathogens circulating in populations (surveillance) and for investigating the similarities and differences between the PFGE patterns of cases and candidate exposures (outbreak investigation)

ISS3 – Translate innovation into practice (Space=Public health and university): 2010–2011

Researchers worked with Utah Department of Health personnel on the following ISS3 deliverables: The ISS3 is a success- or fail-stage for translating Epinome from campus to the public health field. To prepare for a smooth implementation in the ISS3, researchers spent considerable time and resources to replicate the UT-NEDSS IT environment and data structure in the University research environment. With the Institutional Review Board’s permission, the research team obtained access to de-identified data from the UT-NEDSS system, and then, to sustain a realistic dataset, matched the remaining identifiers to an in silico population to provide fictional details where identifiable details had been scrubbed. Utah Department of Health successfully exported the UT-NEDSS data according to the Epinome’s ontology schema. The researchers and practitioners began to integrate Epinome into the state IT system - Establishing operational communication between Epinome and Laboratory Information Management System (LIMS) and UT-NEDSS.

Exported UT-NEDSS data according to the Epinome ontology schema

Established access to the UDOH’s Laboratory Information Management System

Develop UT-NEDSS mirror functioning at UDOH

Establish Apache/Spring server responds to calls

Success in the ISS3 stage requires special attention to be paid to maintaining an effective and active partnership. This stage involves translation of research innovation from university workspace into public health workspace: this workspace crossover is a potential tension point. To address this concern we developed a protocol and shared vocabulary to facilitate communication, provide contextual information, and assure correct and efficient utilization of all resources. Shared identification and articulation of the ISS3 deliverables helped to maintain a mutual understanding assuring successful collaborative innovations across workspaces.

The Epinome partners jointly developed a plan for agile delivery and training, usability testing, and deployment strategies. The researchers also planned to transfer knowledge necessary (e.g. software documents, etc.) to maintain Epinome in practice. Public health was in process of developing a sustainability plan.

At this stage, public health began to realize that the Epinome research product possibly would become an operational resource for their practice and gradually developed a stewardship attitude towards the Epinome. The ISS3 had disproportionate effects on public health practice due to required changes in public health’s workflow, data access, and adjustment to unfamiliar visual analytical methods and data displays. The tasks in the ISS3 are more interdependent across workspaces than those in other stages. Investigators at the university depended upon collaborators at practice to accept, understand, and complete certain tasks before moving forward with next research tasks. However, for public health collaborator, these research tasks were low priorities for their practice and competed with their day-to-day job demands. Key personnel functioned as liaisons were indispensable in overcoming these challenges. The liaisons who understand or have previous experiences in both research and practice, served as information pipelines between the campus and the field. We have avoided pitfalls and tension points due to the liaisons’ constant coordination of competing demands on personnel, communication channels and appropriate assignments for public health personnel.

ISS4 - Evaluate the impact in the “Real World” (Space=University and public health): 2011

There were four ISS4 deliverables which comprised the following: The ISS4 is an impact stage. The UDOH Institutional Review Board approved “opening the public health space” for researchers to evaluate the impact of adopting Epinome. The partners planned to jointly develop an evaluation plan and metrics by analyzing web log files to identify patterns of use and problems experienced while using the software. Additionally, the team can create plans to disseminate the novel discovery, physical product, and its evaluation to research and practice communities.

Collect and analyze Epinome server records (click stream analysis) to assess and describe patterns of usage

Created and evaluated mockups for Epinome special application for foodborne disease outbreak investigation

In summary, the PHI-ISS framework provides a) a priority tool for planning and administrating various tasks and deliverables, b) a new perspective to consider how to allocate resources across organizations at appropriate time, c) a communication facilitator to identify and mitigate unnecessary tension or conflict among partners, and d) a road map to measure the status of a public health informatics research project.

Status of other projects in 2011 relative to Innovation Stage and Space

Figure 4 presents the status of other six research projects conducted by the RMC investigators or informatics graduate students mentored by the RMC faculty from 2007 to 2011. As of the end of 2011, these projects are at various points along the spectrum of Public Health Informatics Innovation Stage and Space. Projects that were the focus of research during the first three years of funding are further along in the spectrum. The two projects currently in Innovation Stage IV were developed and deployed in a single environment allowing other users to access the system using the Web: GermWatch, a pathogen-specific surveillance system, was developed and deployed in the Intermountain Healthcare environment while PHAccess, a secure communication and project management network, was developed and deployed at the Utah Department of Health. In contrast, the two projects currently in Innovation Stage III have required more than one environment to successfully develop and deploy the research output. Epinome has been developed in the research environment but requires integration with data and systems at the Utah Department of Health before it can become operational and impact public health. Similarly, RTCEND, a project transmitting case reports electronically from healthcare to public health, requires implementation in both the clinical and public health environment to realize the exchange of information between the two settings. To complicate the situation further, the public health disease surveillance system (Utah-NEDSS) to which these applications must be linked was developed and deployed during the past few years. The knowledge management project in Innovation Stage II was initiated more recently in 2010 and is currently focused on collaboration to design and develop systems that meet evidence-based public health needs. Finally, while Innovation Stage I for the funded projects occurred during the grant writing period, the ongoing collaborations between public health practitioners and researchers and students in the academic setting have spawned new ideas and prototypes that have the potential to further advance the science and practice of public health informatics. These additional projects are described below.

Figure 4.

Status in 2011 of RMC-related projects along the Innovation Stage and Space spectrum

GermWatch in the Operation/Impact Stage

GermWatch (http://www.germwatch.org) is a pathogen-specific surveillance system that was initiated in 2001 by researchers in the Department of Pediatrics at the University of Utah and supported by a Utah Department of Health’s grant, then further developed with partial support for junior investigators with the RMC. GermWatch is an information resource that provides timely (∼24 hour delay) surveillance information about regional microbiological activity including viral respiratory surveillance, viral and bacterial gastrointestinal infections and antimicrobial resistance (See Figure 5). The system is based on microbiological testing performed in Intermountain Healthcare’s large integrated healthcare delivery network that provides a majority of the healthcare for the state of Utah, which affords the potential to approximate population-based rates. Intermountain Healthcare operates the GermWatch system in partnership with the University of Utah Departments of Pediatrics and Biomedical informatics on behalf of Utah clinicians and state and local health departments. The system is currently being modified to provide information more suitable for consumption by the public.

Figure 5.

GermWatch interface providing a portal to pathogen-specific surveillance and brief messages to guide clinical and public health decision making

GermWatch greatly enhances the breadth Utah’s surveillance system by adding pathogen-specific data based on microbiologic testing in Utah’s largest integrated healthcare delivery system to syndromes based on Emergency Department chief complaints and notifiable diseases reported under State law (6). The system is also novel in the sense that the information provided is specifically geared towards meeting the information needs of healthcare providers and healthcare system administrators, as well as providing information to public health about common outbreaks that are not part of the reportable disease profile (e.g., RSV, adenovirus, enterovirus, human Metapneumovirus) (7). GermWatch data has been used by COE researchers to conduct cutting edge infectious disease modeling research including forecasting RSV epidemics using meteorological variables (8) and modeling seasonal variation of RSV (9). The data on antibiotic resistance is available for the entire system and can be partitioned by conditions and various populations (e.g., ambulatory pediatric urinary tract infections). Ongoing research is addressing how to optimally present and provide access to this data to front line clinicians.

Surveys and anecdotal reports have documented substantial and sustained interest on the part of clinicians in having access to this information because they believe the data to be useful in clinical practice for improving diagnostic accuracy, improving clinical decision making (e.g., antibiotics prescribing, testing for viruses), improving communication with patients/parents about pathogens circulating in their regions(6–8, 10). Administrators charged with managing healthcare resources and systems find the data valuable for making decisions about human resources (e.g., knowing when they will need to call in extra staff), implementing visitor restrictions to reduce nosocomial infections and planning and implementing RSV immunoprohylaxis for high-risk patients [Personal communication PHG]. Public health officials report that the system is a valuable addition to their surveillance arsenal because it provides ready access to information about conditions they are not resourced to track [Personal communication PHG].

PHAccess in the Operation/Impact Stage

During the first three years of COE funding, the research and collaboration efforts due to the “Interact” project resulted in a shared and improved understanding of the problems in communication between clinical and public health settings. For example, we documented problems among urgent care providers with understanding public health reporting requirements and their role in population health (10).Urgent care providers are highly likely to be first line responder during an outbreak. We also identified communication barriers between public health settings. Between the various clinical and public health settings, secure transmission of information was limited to phone calls and faxed paper reports.

In 2008, the development of PHAccess was initiated by a RMC research informaticist who was involved in informatics research and training but stationed in the public health environment. Initially, the purpose of the application was to share information about public health issues with all 12 local health departments in Utah. PHAccess expanded through user input about their needs and has value for public health practitioners and agencies. It allows secure communication about current issues/outbreaks between state and local public health departments and clinical partners. There is an easy single sign-on to access secure applications, such as Epi-issue tracker, ILI surveillance reporting, UT-NEDSS and secure messaging. PHAccess has a simple framework that allows for the ability to create and publish new applications with little development effort. PHAccess allows secure communication between researchers and public health practitioners. Finally, PHAccess has the ability to easily bring on new users.

The value of PHAccess can be measured by usage. As of August 2011, there were a total of 898 Registered Users, including public health practitioners affiliated with the Utah Department of Health (n=470), local and county health departments (n=114), as well as physicians and infection preventionists from hospitals in Utah (n= 138), and other users from state, national or commercial entities (n= 175). As many as 339 epidemiologic issues have been collaboratively addressed with local health departments. Finally, 57,174 secure emails have been sent using PHAccess. PHAccess rapidly transitioned through the stages of the ISS framework and is currently available for adoption by other public health entities.

RTCEND in the Translating Stage

Starting in 2007, RMC investigators from informatics, clinical, and public health settings collaborated to design a system to implement real-time communication of electronic notifiable (aka reportable) diseases (RTCEND). The team was not addressing detection of reportable events, but rather addressing the content and structure of electronic messages for a clinical setting to report diseases such as hepatitis A or salmonella to public health. Investigators collaborated (in Stage 2) to understand the public health workflow and information needs, and evaluated existing HL7 and CDC messaging standards (11, 12). Based on evidence derived from the investigations and the existing messaging infrastructures, a proposed Health Level 7 (HL7) version 2.5.1 message format was defined that was extendable to accommodate any reportable condition and allowed for the inclusion of both laboratory and clinical information in a message (see Figure 7). In addition to local collaboration efforts, RMC investigators collaborated with the CSTE/CDC case report standardization workgroup (CRSWg) to provide input and be informed by the CSTE policy concerning the recommended core content of a case report.

Figure 7.

RTCEND Reporting Process

Currently, RTCEND is in Innovation Stage 3 and becoming operational by Intermountain Healthcare and the Utah Department of Health. Prior to implementing the system, researchers evaluated the quality of electronic reporting using RTCEND compared with traditional manual reporting methods, and focused on the timeliness, completeness of information content in the initial report, and completeness of transmitting case reports for recognized reportable events.(13)The prospective evaluation (performed in July 2010) and the retrospective evaluation (performed on messages sent from October 2010 to February 2011) found that electronic messages were more timely than paper reports sent from other healthcare facilities (p<0.0001) (13). The HL7 messages also included more complete information when compared to the content of paper reports from other facilities, particularly concerning hospitalization status, and the reporting contact’s name and phone number (13).

RTCEND illustrates the challenges of implementing new informatics innovations in the operational public health environment. During the time of this project, several important changes occurred in the public health environment that required modifications to the implementation plans. To complete the research and evaluate impact on public health practice, the Utah Department of Health must be able to receive and integrate messages into their surveillance system. During the course of this research, UDOH has developed and implemented a new Utah-NEDSS system with an outside contractor (TriSano) and the reporting standards have changed. The RTCEND message format is currently being modified to meet the intent of RTCEND to send clinical information while fulfilling the recent Meaningful Use requirements defined by the HITECH Act to send electronic laboratory messages (14). Electronic messages are currently being sent from 22 Intermountain Healthcare facilities to a test environment at UDOH. The messages will need to be integrated into the Utah-NEDSS before their impact on public health work flow and disease control can be assessed. Once integration has been completed, Intermountain Healthcare will explore enhancing the messages to include other clinical data requested by public health.

Knowledge Management in the Collaborative Stage

In the fall of 2009, research was initiated to address the management of public health knowledge required to improve communication between public health agencies and their clinical and laboratory partners. The research is focused on the following three use cases:

Public health reporting from laboratory and clinical settings,

Public health notifications about alerts and “what’s going around”, and

Request and response for information to support a public health investigation, particularly using structured reports (e.g. based on the HL7 clinical document architecture).

The goal is to demonstrate a new model for managing public health knowledge using service-oriented architecture (SOA) and standard terminologies that a) allow public health authorities to author, store, and ‘publish’ computer-interpretable knowledge, and b) allow users to access the knowledge using context-aware information retrieval strategies, view human-readable content, download structured content using web services for execution within their own systems, and subscribe or query for updates. This project is a reference implementation to assess the feasibility and value of the proposed system.

Currently, we are focused on the public health reporting use case in Innovation Stage II. Researchers and practitioners are jointly assessing the problems and defining requirements in the public health practice Space to ensure that design and development efforts underway in the research Space are guided by evidence. To determine content and functional requirements, researchers reviewed existing knowledge resources and analyzed business process (15). We surveyed hospital and commercial laboratories to describe current processes, estimate their burden to comply with public health reporting, and to evaluate the business need and readiness for a service using SOA to deliver standardized reporting specifications. We are using ethnographic methods to get feedback from public health, clinical, and laboratory users on design and workflow issues.

To develop the system, we are using the infrastructure developed for the University of Utah’s Federated Utah Research and Translational Health electronic Repository (FURTHeR). The system uses Web services developed in JAVA for intersystem communication and requires no proprietary tools by laboratory or hospital personnel/systems. The knowledge is stored in XML files located in a metadata repository. The terminology services are handled by Apelon’s Distributed Terminology System (DTS) (http://apelon-dts.sourceforge.net/) which supports national data standards. We are using Altova’s XML Spy and StyleVision to create data models and data entry forms, but these tools are not needed by system users. The knowledge is accessed through a query interface that allows a user to specify a condition, jurisdiction and role (e.g., laboratory or clinician). The query results are returned by a service and displayed.

To develop the system, we are using the infrastructure developed for the University of Utah’s Federated Utah Research and Translational Health electronic Repository (FURTHeR). The system is Web-based, using Web services and developed in JAVA, and requires no proprietary tools by laboratory or hospital personnel/systems. The knowledge is stored in XML files located in a metadata repository. The terminology services are handled by Apelon’s Distributed Terminology System (DTS) (http://apelon-dts.sourceforge.net/) which supports national data standards. We are using Altova’s XML Spy and StyleVision to create data models and data entry forms, but these tools are not needed by system users. The knowledge is accessed through a query interface that allows a user to specify a condition, jurisdiction and role (e.g., laboratory or clinician). The query results are returned by a service and displayed.

We will test system usability and evaluate functionality during Innovation Stage 2 by exporting the reporting specifications for Utah and using the knowledge to inform public health reporting from the University Healthcare enterprise data warehouse. Further development is limited by reduced support for the RMC.

Pilot Projects in the Initial Stage

A center of excellence in public health informatics requires the ongoing infusion of new ideas and new investigators to address emerging and future public health informatics needs. Collaborations forged by the RMC environment have spawned a variety of new research pilots conducted by students and other investigators interested in public health informatics related problems. Two illustrative examples currently underway are shown in Figure 4. As public health practitioners and researchers engage in discussions about problems they face and research in progress, students and investigators identify a) new and interesting research questions, and b) opportunities to apply informatics methods used in other domains to the public health practice space. For example, in the Spring of 2010, a graduate informatics student with a background in engineering identified a solution for automating the analysis of reporting logic defined in CSTE position statements (16). The student developed a JAVA-based tool with a user-friendly excel file for input to test reporting logic for hepatitis (17). To lay down a solid base for this research initiative, the RMC research faculty guided the student to solicit input from three public health epidemiologists that authored the hepatitis-related position statements. The epidemiologists provided feedback about the tool and received follow-up reports about the logic being revised for re-balloting in 2011. However, more input from public health and research resources would be required to move it from the Innovation Stage I to Innovation Stage II.

Similarly, another graduate informatics student is developing a grid-based tool to improve the processing of cause of death narratives to enable real-time monitoring of deaths due to pneumonia and influenza and other events with public health implications (18). While this research is not funded by the RMC, its innovation journey and impact are highly relevant to the RMC’s innovation research and integral to the mission of developing sustainable partnerships and researchers that understand public health practice.

Other impacts of partnerships fostered by public health informatics innovations

The sustained collaboration between partners resulted in other benefits that may not be obvious to outside observers. The collaborations resulted in “floating everyone’s boat”, being a force magnifier for training, engaging partners in the national standards development efforts, and improving the analysis of problems in the public health and clinical environment. For example, first, the collaboration between public health practice, research and academics resulted in a)knowledge transfer in both directions, b) integration of practitioners into the national efforts (ELR), c) creating a ‘safe environment’ for public health practitioners to get up to speed in a faster way; and d) developing a model for informatics students to participate in “Infoaids” during public health crisis situations.

Second, the COE was a force magnifier for training the public health workforce, informatics graduate students, and junior investigators. For example, public health collaborators had access to continuing education and in turn contributed as faculty for the AMIA 10x10 course. Graduate informatics students became engaged in research leading to careers in public health informatics and had the opportunity to interact closely with public health practitioners, evaluate surveillance systems and CSTE Position Statements, and make significant contributions to the CDC/CSTE Case Report Standardization Workgroup (19). Five junior faculty investigators in University of Utah jump-started careers built on the grants opportunities and collaborations fostered through the RMC.

Third, improved problem analysis was demonstrated during theH1N1 outbreak in 2009. Prior to establishing the RMC, many of the key players who would need to respond to this type of public health events did not know one another. As the H1N1 outbreak evolved, researchers from the center were able to rapidly and systematically evaluate real communication in the field and identify problems with duplication of effort and communication overload (20). After the first wave of the outbreak, a new communication strategy was developed in large part due to the ongoing partnership among the major stakeholders, many of whom were represented in the RMC partnership (Figure 7). The revised organizational communication strategy included a taskforce to coordinate messaging and deliver a unified public health messages through chief medical officers with health care entities, and requirements were identified for future message delivery to improve response to public health threats.

Finally, there are other tangible and intangible impacts resulting from the collaborations fostered by the RMC. Practitioners and researchers from the RMC jointly participated in national standards development efforts to increase their understanding of standards and speed up the innovation process at the home organziation.Sharing of ideas were also fostered through the RMC weekly virtual meetings to discuss new research, issues, and outside innovations. There was a total of 164 papers, posters, abstracts, presentations, and white papers published during the past 5 years. The collaborations reached outside of our own RMC to establish the Community of Innovators in Epidemiology and Public Health Informatics (coi-EPHI).

Conclusion and Discussion

The Rocky Mountain Center of Excellence (COE) in Public Health Informatics’ collaborative trajectory provides live examples of our innovation processes within the past five years. Respecting each other’s working space is crucial for successful collaboration between researchers and practitioners. Understanding the innovation stage advances the innovation management across the spaces. Acting indifferently to needs and expectations across workspaces may hamper or even dissolve the collaboration. Sometimes, workspace crossover may cause miscommunication and friction among collaborators. Applying the Public Health Informtics Innovation Stage and Space framework to collaborative activities can help reveal potential challenges early. With mutual understanding of a common framework, we developed strategies to help project managers to anticipate potential points of difficulty and proactively reduce and mitigate potential risks for partnerships. Understanding the boundary and process of practitioner-participated research significantly improved efficiency of public health informatics innovations in Utah.

The seven informatics researches described above also demonstrate that COE is not just a one-time collaboration of distinct research projects within one grant. The people, partnership, shared vision, and mutual understanding and appreciation developed over a long period of time are the core and base for ongoing effective innovations and its successes.