Defining information needs for public health systems and services research.

BACKGROUND: People who lead or manage public health agencies have multiple needs for information in order to do their jobs effectively. In seeking to investigate "what works" in public health practice, investigators in the field of public health systems and services research (PHSSR) have largely overlapping information needs but often require a greater detail, specificity, or comprehensiveness than is routinely available in public health data systems. PHSSR DATA NEEDS MEETING: On April 24, 2014, the PHSSR Center of the University of Kentucky and AcademyHealth convened a 1-day meeting of public health practitioners and PHSSR investigators to identify PHSSR information needs. Meeting participants considered data needs for three PHSSR domains: the organization of public health agencies and services, the use of rapidly evolving health information technologies, and the financing and economic evaluation of public health activities. FUTURE DATA NEEDS: Identifying data needs in these and other PHSSR domains requires clarification of research questions, consideration of research methods, a balance of imagination and practicality, and investments to extend the information captured in existing administrative, financial, and population health monitoring systems.

People who manage public health agencies and implement public health programs and policies have diverse needs for information in order to do their jobs effectively.1–3 This includes information about the nature and health of the populations they serve, the physical and cultural environments where those populations live and work, the performance and impact of their programs and policies, the capacity of their workforces, their uses of human and financial resources, and the cost and effectiveness of various options available to them in addressing public health problems. The field of public health systems and services research (PHSSR) also needs a spectrum of information in order to develop an evidence base that can guide public health practice and policy making. PHSSR seeks to answer a deceptively simple question: what works in public health? Having answered that question in a particular place or situation, attention must be directed to a second question: can that knowledge be translated and implemented effectively in other settings?4–7

Beyond targeted research studies that use experimental or other controlled methods to test the impact or economic value of particular interventions, typically by comparing intervention and referent groups, there is also a need for broader observational studies that can discern the impact and value of public health systems, services, programs, and policies in more general practice. In a sense, public health practice represents a daily, ongoing constellation of “natural experiments,” as public health agencies at various levels and in different places go about their jobs of setting priorities, establishing plans and policies, implementing programs, providing services, and responding to novel or urgent situations as they arise. As a discipline, PHSSR seeks to exploit the learning opportunities that arise from these variations in order to identify the most effective approaches to solving public health problems and to using public health resources.

On April 24, 2014, the PHSSR Center of the University of Kentucky and AcademyHealth convened a 1-day meeting of public health practitioners and PHSSR investigators to identify PHSSR information needs. In preparing for this meeting, we predicated our planning on four assumptions:

There is a strong kinship as well as a distinction between the information needs of public health practice and PHSSR. For example, the information that a health officer needs to direct a state or local health department is both similar to and different from the information that a PHSSR investigator would need to study what makes that health department or its programs more or less effective. Ideally, if a researcher is successful in generating answers to such questions for a particular health department, those answers would become part of the evidence base available not only to that health department but also to others.

The information required to describe public health services and expenditures and to evaluate public health programs and policies occupies much of the overlap between these two domains. Developing a clear understanding of this overlap would be important to improving data resources for PHSSR.

The task of identifying the data needed to address these and other PHSSR questions will be influenced by the nature of the research methods that would be applied, and we might envision two strategies for approaching this task. The first would be to define the ideal methodologic approaches for addressing the various research questions and then the types of data that would be required to apply these methods. The second would be to consider the nature of data that are currently available or possible to obtain in the near term, which would then influence the methodologies that could be applied. While recognizing that our focus was on identifying data needs and not research methods, we anticipated that our deliberations would fluctuate between these two strategies, requiring a balance of imagination and practicality.

To define information needs for PHSSR it is necessary to define the questions we want to answer, a task that must be done within the context of specific public health domains. In general, we agreed that it is preferable to ask questions first and find the data second, rather than vice versa.

Regarding the fourth planning assumption, we elected to focus our discussion of data needs in three important domains of PHSSR:

Anyone who has worked in public health has undoubtedly wondered why some health departments are seemingly stronger or weaker than others. Is it their leadership, their organizational structure, their workforce, their level of funding or political support, the dynamics of their communities, the strength of their relationships with other agencies and organizations, the terms or interpretation of the state or local laws that define their authorities, or some combination of these and other factors? And, by the way, how do you measure the effectiveness of a health department and how can you relate the attributes and actions of a health department or a particular program to trends in the health of a community?4–7

Public health agencies have long depended on information that arises from the use of healthcare to monitor population health and on information from various population health surveys. Healthcare information systems are increasingly being automated, and the massive federal investment to incentivize the “Meaningful Use” of electronic health records (EHRs) and electronic health information exchange includes explicit public health objectives, including requirements that clinicians use their EHRs to share certain information electronically with state or local public health authorities.8 In addition, the widespread use of online social networks,9 the growing availability of Internet-based sampling and recruiting methods to conduct population surveys,10 and expanding efforts to explore “big data” information resources11 all provide new opportunities for monitoring a spectrum of population health indicators. Local, state, and federal public health agencies are focusing substantial attention on harnessing new information technologies to meet their information needs and enhance their capacities.

For PHSSR critical questions will include: What are the optimal approaches to incorporating new information technologies into public health practice? How can we demonstrate the impacts these technologies have on public health programs and services or population health outcomes? How can we assure that the public health workforce is able to take full advantage of new technologies? Which public health agencies will be able to make the most effective use of new information opportunities and why? Will the population health objectives of the Meaningful Use program be achieved, i.e., what can and should be done to assure that Meaningful Use is meaningful for public health?

For example, the Office of the National Coordinator for Health Information Technology has characterized the aims of the three stages of the Meaningful Use as adoption of certified EHR technology (Stage 1), routine, ongoing, and automated exchange of health information (Stage 2, in progress), and demonstration of improvement of health outcomes (Stage 3). Clearly, understanding the determinants of progress in achieving the public health benefits of these transitions, particularly between the second and third stages, will be a critical domain for PHSSR.

Public health agencies must be able to describe and monitor how they allocate and use their financial resources in ways that can be connected transparently to the services they provide.12 Ideally, those expenditures could be linked to health outcomes among populations, providing a “feedback loop” to inform future allocations. To date this daunting task mainly falls within the domain of PHSSR, reflecting the state of public health accounting systems and the methodological challenges inherent in relating public health spending to population health outcomes.13 Better financial data will be needed to characterize public health expenditures as a prerequisite to economic research that aims to weight the benefits or value that results from those expenditures. A substantial proportion of public health funds are spent on sustaining public health infrastructures, such as the development and maintenance of information systems, emergency preparedness capabilities, and general administrative services. Thus, in addition to assessing the value of specific programs or services, a particular challenge will be the need for data and approaches for weighing the value of infrastructure investments.

As demonstrated by the accompanying reports that address PHSSR data needs for each of these PHSSR domains, our deliberations integrated consideration of the questions we want to answer in PHSSR, the interplay between the research methods that could be applied and the nature of potential data resources, and our ability to think beyond current information resources yet arrive at recommendations for developing new data sources that are within reach to answer these and other key questions.14–16 In hindsight, our planning assumptions were largely but not completely fulfilled. With respect to the anticipated overlap and distinction between data needs for public health practice and PHSSR, our observation was that the overlaps were vastly greater than the distinctions. The distinctions were less of substance and more of degree. In other words, for PHSSR purposes, an investigator is likely to need the same types of information that a public health manager would need and ideally have, although the level of detail, specificity, and comprehensiveness is likely to be greater for PHSSR than what is routinely feasible in practice. As suggested by the accompanying papers, this would require a variety of strategies to develop data resources for PHSSR, including enhancements to routine administrative, financial, or health monitoring systems, perhaps by focusing in targeted but representative locations.