Enterprise Evaluation: A New Opportunity for Public Health Policy.

Standard evaluation practice in public health remains limited to evaluative measures linked to individual projects, even if multiple interrelated projects are working toward a common impact. Enterprise evaluation seeks to fill this policy gap by focusing on cross-sector coordination and ongoing reflection in evaluation. We provide an overview of the enterprise evaluation framework and its 3 stages: collective creation, individual data collection, and collective analysis. We illustrate the application of enterprise evaluation to the Gulf Region Health Outreach Program, 4 integrated projects that aimed to strengthen health care in Louisiana, Mississippi, Alabama, and the Florida Panhandle after the Deepwater Horizon oil spill. Shared commitment to sustainability and strong leadership were critical to Gulf Region Health Outreach Program's success in enterprise evaluation. Enterprise evaluation provides an important opportunity for funding agencies and public health initiatives to evaluate the impact of interrelated projects in a more holistic and multiscalar manner than traditional siloed approaches to evaluation.

Communities along the US Gulf Coast are geographically and medically vulnerable.1–4 Health inequities have plagued the region for decades.5,6 Hurricanes Katrina and Rita in 2005 exacerbated an already fragile situation and highlighted the area's existing public health and health care deficiencies.7–10 Despite significant public health infrastructure investments, the 2010 Deepwater Horizon oil spill revealed that residents of the most impacted Gulf Coast communities still lacked sufficient access to effective primary and specialty care for a range of physical, mental, behavioral, and environmental health concerns.11 After the spill, the civil and criminal monetary penalties resulting from violations of the major environmental policies (eg, Clean Air Act, Clean Water Act, Oil Pollution Act) primarily targeted natural resource restoration.12 Claims settlements compensated for economic impacts and damages to individuals.13–15 Notably, both the environmental penalties and the claims processes failed to account for the public's health.12,16 An important and unique opportunity to address this policy gap and improve health disparities in the Gulf region was presented by the Deepwater Horizon Medical Benefits Class Action Settlement.

The Gulf Region Health Outreach Program (GRHOP), a component of the Deepwater Horizon Medical Benefits Class Action Settlement, was conceived to address many of the long-standing health inequities in Gulf Coast communities. GRHOP consists of 4 integrated projects that aim to strengthen health care in coastal communities in Louisiana, Mississippi, Alabama, and the Florida Panhandle. The program is supervised by the court and is funded with $105 million from the Medical Settlement. The target beneficiaries of GRHOP are residents, especially the uninsured and medically underserved, of 17 coastal counties/parishes in the 4 states. GRHOP is the first program to emerge from an environmental tort that seeks to bolster human health in the affected communities without restricting activities to address the health issues directly linked to the adverse event.17

In this article, we briefly describe GRHOP and then discuss efforts to evaluate the program's long-term collective public health impact through the implementation of an enterprise evaluation. Enterprise evaluation combines collective assessment with individual program evaluation to evaluate the collective impact of multiple related programs with a common goal. This framework counters standard public health policies and practices by deliberately incorporating cross-project coordination and collective impact measures into implementation and evaluation. GRHOP's application of enterprise evaluation provides an important example of how public health practice can influence policy, highlighting the ways in which policy and practice are interrelated and mutually reinforcing.18

GRHOP

GRHOP seeks to inform community members about their health and provide access to skilled frontline health care providers supported by specialists in physical, environmental, behavioral, and mental health. The program consists of the Primary Care Capacity Project (PCCP), the Environmental Health Capacity and Literacy Project (EHCLP), the Mental and Behavioral Health Capacity Project (MBHCP), and the Community Health Workers Training Project (CHWTP) (Table 1). It also incorporates a community involvement component. A coordinating committee ensures that the projects function in a cooperative and integrated manner, with sufficient flexibility to adjust implementation to respond to changing needs and circumstances.

TABLE 1

The Integrated Projects of the Gulf Region Health Outreach Program

Name of Project	Description	Funded Organization
Primary Care Capacity Project	Expands access to high-quality, sustainable, community-based primary care, focusing on the development of primary care linkages to specialty mental and behavioral health care, as well as environmental and occupational health services.	Louisiana Public Health Institute, New Orleans, Louisiana
Environmental Health Capacity and Literacy Project	Builds environmental health capacity to deliver coordinated specialty care, integrates CHWs with training in environmental health issues into primary care clinics, expands maternal and child health services in communities experiencing repeated disasters and environmental exposures, and exposes high school students to hands-on environmental health activities and research experiences.	Tulane University, New Orleans, Louisiana
Mental and Behavioral Health Capacity Project	Provides mental and behavioral health treatment in the short-term and long-term supportive services to improve the overall well-being of individuals, families, and communities affected by the DWH incident.	University of West Florida, Pensacola, Florida; University of South Alabama, Mobile, Alabama; The University of Southern Mississippi, Hattiesburg, Mississippi; Louisiana State University, New Orleans, Louisiana
Community Health Workers Training Project	Trains CHWs who help residents navigate the health care system and access needed care.	University of South Alabama, Mobile, Alabama
Community Involvement	Coordinates community involvement and outreach efforts in each of the 4 GRHOP-eligible states.	Alliance Institute, New Orleans, Louisiana

Abbreviations: CHWs, community health workers; DWH, Deepwater Horizon; GRHOP, Gulf Region Health Outreach Program.

Enterprise Evaluation

The integrated orientation of GRHOP promotes collaboration across projects, challenging the traditional approach where leaders within a program portfolio largely work independently and come together during isolated instances. GRHOP's orientation toward cross-sector collaboration and the desire to maximize the collective impact of its projects led us to utilize a novel approach to evaluation we call enterprise evaluation. Enterprise evaluation is a practical framework that measures collective outcomes across multiple projects. Developed in collaboration with GRHOP partners and an external evaluation consultant, enterprise evaluation emphasizes cross-sector coordination, collective outcomes, and ongoing analysis, filling an important policy gap in public health evaluation. Although funding agencies in environment and health increasingly emphasize the need for collaborative and transdisciplinary approaches, standard evaluation practice remains limited to evaluative measures linked to individual projects, even if multiple interrelated projects are working toward a common impact.19,20 This traditional approach is evident among public health and environment funding agencies, as well as private foundations.21–26 By restricting the focus of evaluation to individual projects, these program policies limit the ability to examine collective outcomes in a holistic, multiscalar manner.20

In contrast, the enterprise evaluation framework draws from collective impact and developmental evaluation principles to create a coordinated evaluation process among networked projects that fosters collective analysis. Collective impact is a model of cross-sector collaboration aimed at addressing large-scale complex change, with emphasis on aligning project activities and establishing shared metrics for desired collective outcomes.27 Collective impact requires participants to have ongoing and purposeful communication, mutually reinforcing activities, a common agenda with aligned project outcomes, agreement on the ways success will be measured and reported, and a backbone organization to support efforts.20 Backbone organizations can take many forms, including nonprofit organizations, multiple organizations working together, funding agencies, or government.28 Developmental evaluation also emphasizes cross-sector collaboration and systems change and places a strong emphasis on fostering innovation through evaluation, examining the systems in which a program is being implemented, and leveraging ongoing reflection to continuously inform and improve program implementation.29 Developmental evaluation and collective impact both highlight how programs contribute to a given outcome. An outcome cannot be attributed to a specific program in a complex system where there are many potentially influential factors.20,29

Enterprise evaluation involves 3 phases—collective creation, individual data collection, and collective analysis—to assess the achievement of collective outcomes across multiple related, yet functionally independent, projects (Figure 1). As collaborative, networked projects are increasingly the preferred model for public health programs by funding agencies, GRHOP's experience with enterprise evaluation provides an important example of a viable approach for effectively and comprehensively evaluating collective impact.

FIGURE 1

Enterprise Evaluation Approach

Abbreviations: AL, Alabama; CHWTP, Community Health Workers Training Project; CI, Community Involvement; EHCLP, Environmental Health Capacity and Literacy Project; FL, Florida; LA, Louisiana; MBHCP, Mental and Behavioral Health Capacity Project; MS, Mississippi; PCCP, Primary Care Capacity Project.

Collective creation: Development of GRHOP's enterprise evaluation framework

Since GRHOP projects were designed interdepen-dently,30 individual projects shared overarching objectives and often worked with the same communities and organizations. To achieve GRHOP's intended impact to strengthen health care in the communities most affected by the oil spill, the projects needed to work collaboratively to effect change at various levels, including the client, clinic, system, community, and population. In addition, many projects established contracts or partnerships with the same health clinic operators. For example, 1 federally qualified health center in Mississippi was actively involved in PCCP, EHCLP, MBHCP, and CHWTP. The overlap in objectives and sites thus created both an opportunity and an imperative for joint project implementation and evaluation.

Each GRHOP partner proposed an evaluation plan as part of its application to the court, but an overall evaluation framework was not part of the settlement. However, during the initial implementation period that focused on starting individual projects, there was a strong commitment from the group to evaluate the program's collective impact. To this end, GRHOP evaluation subcommittee, comprising staff from each of the projects as well as coordinating committee members, was formed and met at least 3 times per year, generally in conjunction with the quarterly meetings that partners were required to attend (Figure 2). Subcommittee meetings initially focused on sharing project-based evaluation strategies and data collection plans. Given the variation in partner activities and the diversity among the proposed data collection plan, subcommittee members engaged an evaluation consultant to assist in developing a joint enterprise evaluation framework, including logic models and short-, mid-, and long-term measures that reflected both overall GRHOP efforts and individual project activities (Figure 3). Short-term outcomes described the changes that were expected to occur in the early phases of project implementation; these changes were often at the level of individual clients and clinic sites. Short-term outcomes were largely specific to each project, whereas midterm outcomes described the first level of joint outcomes across GRHOP projects, though not all projects contributed to every midterm outcome. Likewise, the collective midterm outcomes did not necessarily reflect the full scope of individual project outcomes. Midterm outcomes were more likely to occur at the level of the clinic system and/or community, whereas long-term outcomes were more relevant to populations and clinic systems.

FIGURE 2

Gulf Region Health Outreach Program Enterprise Evaluation Timeline

Abbreviations: GRHOP, Gulf Region Health Outreach Program; TBD, to be determined.

FIGURE 3

Gulf Region Health Outreach Program Enterprise Logic Model

Abbreviations: CHWTP, Community Health Workers Training Project; CI, Community Involvement; EHCLP, Environmental Health Capacity and Literacy Project; MBHCP, Mental and Behavioral Health Capacity Project; PCCP, Primary Care Capacity Project.

To understand how individual GRHOP projects influenced the joint medium- and long-term outcomes, project-specific “zoom-in models” were developed. Each zoom-in model included the same mid- and long-term outcomes from the enterprise logic model, though short-term outcomes were project-specific and more detailed than those included in the enterprise model. Project contributions to mid- and long-term outcomes were highlighted in each model to illustrate that project's pathway from activities to long-term outcomes. All zoom-in models underwent numerous reviews, accounting for available data. The MBHCP zoom-in was the most challenging to develop as it reflected the work of independent partners in the 4 states, which had substantial variation in activities, discrete budgets, and leaders representing different mental and behavioral health disciplines.31,32 Thus, the MBHCP initiative operated as 4 state-specific projects nested within a larger MBHCP enterprise. Each of the 4 GRHOP projects also developed output and process measures for their appropriate short-, mid-, and long-term outcomes. Once the zoom-in models were created, the evaluation subcommittee compiled measures for each long-term outcome based on data collection across projects. The collective creation of a shared logic model required extensive collaboration among partners and, as a result, partners gained full perspective on the varying project contexts and objectives as well as the overlapping project sites and stakeholders. This process resulted in a deeper sense of shared mission among project leaders and fostered additional collective activities to enhance health care capacity across GRHOP sites.31

Individual data collection: Examples from GRHOP's enterprise evaluation

Table 2 highlights examples of several projects' individual evaluation efforts, drawing from PCCP, EHCLP, and MBHCP in Louisiana, Mississippi, and Alabama. The data presented in this table do not capture the full scope of data collected by each of the projects. Rather, these examples illustrate a small portion of the qualitative and quantitative data sources that contribute to 2 of the midterm outcomes and the same long-term outcome.

TABLE 2

Gulf Region Health Outreach Program Individual Project Contributions to a Long-term Outcome: Community Has Greater Capacity for Quality Health Care

Individual Project Activity	Data Source	Results	Collective Midterm Outcome
Primary care capacity investments (PCCP)	Data for 12 clinic operators in Louisiana, Mississippi, Alabama, and Florida that were part of PCCP from 2012 to 2016 were reviewed using Uniform Data System (UDS) measures.	23% increase in the total number of medical visits among 12 clinic operators from 2012 through 2016. 32% increase in the total number of patients among 12 clinic operators from 2012 through 2016.	Stronger health care system (via clinic and GRHOP collaborators)
Placement of CHWs in FQHCs and community-based organizations (EHCLP)	Semistructured interviews (n = 42) were conducted in 2017 with CHWs and CHW supervisors from 18 organizations where CHWs were placed in Louisiana, Mississippi, Alabama, and Florida.	Reported barriers to health care access for clients were lack of insurance, financial issues, lack of transportation, and language barriers. Most participants reported that health care access among clients improved over the past 5 years due to ○ increased availability of insurance; ○ increased number and capacity of FQHCs; ○ increased focus on health prevention and community health literacy; and ○ CHWs' ability to reduce certain barriers to health care access CHWs improved clients' access to health care due to ○ raising awareness about services at the CHW's own organization and in the community; ○ assisting in navigating the health system and health insurance; ○ scheduling and accompanying clients to medical appointments; and ○ translating/interpreting for nonnative English speakers	Stronger health care system (via clinic and GRHOP collaborators)
Providing and enhancing supplemental and supportive MBHCP services and educational opportunities (MBHCP-LA)	Five Louisiana primary care clinics where MCHCP-LA worked were assessed for integration of MBH services for adults and children and MBH administrative integration. Data were retrospectively collected for 2013 and 2016 using a modified version of the Patient-Centered Integrated Behavioral Health Care Principles and Tasks Checklist.	Statistically significant increase in integration from 2013 to 2016 (P < .001) on patient-centered care, population-based care, measurement-based targeted treatment, evidence-based care, and accountability. MBH services moved from siloed (level 1) MBH delivery to an embedded midpoint (level 3) toward fully integrated services (level 5).	Specialty health, environmental health, and MBH services and/or referral systems are embedded into primary care and/or community settings
Providing and enhancing supplemental and supportive MBHCP services and educational opportunities(MBHCP-MS)	Client health records from 2013 to 2017 from 1 FQHC system in Mississippi where MCHCP-MS worked were reviewed to examine changes in MBH services provision.	5 clinics were served between September 2013 and August 2014 and 9 clinics served between September 2016 and August 2017, an 80% increase in access points for MBH services. There was a 167% increase in full-time behavioral health providers (BHPs) between April 2013 (n = 3) and April 2017 (n = 8). An average of 85 monthly warm handoffs were performed between September 2013 and August 2014. An average of 180 monthly warm handoffs were performed between September 2016 and August 2017, a 112% increase.	Specialty health, environmental health, and MBH services and/or referral systems are embedded into primary care and/or community settings
Providing and enhancing supplemental and supportive MBHCP services and educational opportunities(MBHCP-AL)	BHPs in 5 clinics in Alabama quantified their daily activities and interactions, including number of warm handoffs from a primary care provider to a BHP. BHPs completed an online project-created tool at the end of each work day, which took an average of 90 s to complete.	At the start of the project in 2012, none of the 5 clinics in Alabama provided integrated MBH services. The baseline number of warm handoffs per month was 0. By June 2016, all 5 clinics were engaged in integrated health care and had an on-site BHP as part of their health care team. Between 2012 and 2016, the number of patients referred to BHPs via a warm handoff from the primary care provider increased from 0% to 43.3%. By June 2017, 66% of new patients referred to BHPs were referred through a warm handoff.	Specialty health, environmental health, and MBH services and/or referral systems are embedded into primary care and/or community settings

Abbreviations: BHPs, behavioral health providers; CHWs, community health workers; EHCLP, Environmental Health Capacity and Literacy Project; FQHCs, federally qualified health centers; GRHOP, Gulf Region Health Outreach Program; MBH, mental and behavioral health; MBHCP, Mental and Behavioral Health Capacity Project.

Since data collection for each project was completed independently, there was the potential for repetition in the data collected or having separate data collection efforts underway at the same site(s). In addition, distinct approaches to data collection could impede GRHOP's ability to compare data across projects. However, independent data collection provided benefits outweighing these challenges. Independent data collection allowed for enhanced intraproject coordination of resources and scheduling, which was particularly important considering the complexity and geographic dispersal of partners and sites. Individual projects were most familiar with the appropriate stakeholders to involve in evaluation efforts and had expertise for selecting or creating data collection tools. Furthermore, individual projects measured outcomes according to each project's zoom-in model, thus ensuring that data collection was still informed by a collaborative effort to measure GRHOP's impact.

Collective analysis: Utilizing enterprise evaluation data to measure GRHOP's collective impact

After individual data collection is completed, the next step in the enterprise evaluation process is to examine how data collected by individual projects contribute to collective impact. While fully examining the collective impact of GRHOP is outside the scope of this article, we can provide some examples of GRHOP's progress toward achieving collective objectives. As highlighted in Table 2, multiple projects contributed to the same midterm outcomes; PCCP and EHCLP data both relate to a stronger health care system, and the data from MBHCP partners in Louisiana, Alabama, and Mississippi relate to how MBH services and/or referral systems were embedded into primary care settings. By contributing to common midterm outcomes, these individual evaluation efforts provide a multidimensional understanding of GRHOP's impact.

Individual data collection efforts undertaken by PCCP and EHCLP were complementary. The EHCLP's findings that clients have better health care access today than 5 years ago are corroborated by PCCP's data regarding the increase in medical visits (and thus services provided) and unique patients seen. Whereas PCCP's data capture the higher number of visits and patients, EHCLP's data pertain to perceptions of health care providers regarding improved health care access and the reasons behind improved access. In this sense, EHCLP's data provide a nuanced understanding of how the health care system has been strengthened (eg, increased availability of insurance, increased number and capacity of federally qualified health centers in the area). Similarly, PCCP's data describe changes occurring within the health care system at a larger scale than EHCLP's data, which were collected from a subset of the health care workforce (community health workers and community health worker supervisors). Accordingly, the type of data collected by PCCP complements EHCLP's data by providing insights into the overall health care landscape. Finally, PCCP and EHCLP evaluated different dimensions of a strengthened health care system. While PCCP's data provided quantitative evidence of a strengthened health care system in terms of health care utilization, EHCLP's data provide qualitative evidence of the same outcome in terms of health care access. By examining the health care system from both provider (ie, EHCLP) and clinic operator (ie, PCCP) perspectives, GRHOP partners gain a holistic and robust view of the health care system and the ways it changed. In accordance with collective impact approaches to evaluation,20 our collective analysis does not assert a causal linkage but rather shows potential contributions of these individual projects to strengthened health care systems.

Similarly, the data collected by MBHCP highlight indicators of embedded and integrated mental health care in primary care clinics across states and how they were measured in project-specific ways. All MBHCP projects collected data about the number of clinics offering MBH services and the number of integrated MBH providers placed within those clinics before and after project implementation. As illustrated in Table 2, MBHCP in Alabama and Mississippi also measured the number of patients referred to MBH services through a warm handoff from the primary care provider to the MBH provider while the patient was in the clinic. This referral pathway is a pivotal process in integrated care since it reduces no-show rates, which enhances sustainable billing; improves access to specialty care; may combat mental health stigma; and may reflect the extent to which MBH providers are seen as important contributors to patient wellness and as peers on the health care team.33 The MBHCP data collected in Louisiana highlight the organizational components that contribute to fully embedded services and involved an assessment of changes in perceptions among clinic administrators, MBH providers, and coordinators regarding patient-centered care, population-based care, measurement-based targeted treatment, evidence-based care, and accountability. These organizational aspects are aligned with national models for fully integrated care. While the MBHCP projects utilized individualized data collection systems to evaluate project impact, these examples, similar to those from PCCP and EHCLP, illustrate how the enterprise evaluation model has the potential to elucidate not only individual project outcomes but also amplified collective impact. All tell the story of a stronger health care system and increases in embedded mental and behavior health care in clinics primarily serving vulnerable patients.

Further reflection on collective impact will be the focus of a program-wide workshop to be held when all project activities wrap up. The workshop will facilitate collective analysis of individual data collection, focusing on key findings from individual projects and assessing the extent to which GRHOP resulted in the intended collective midterm outcomes. While it is unlikely that long-term outcomes will be fully achieved in GRHOP's time frame of 6 years, the workshop will provide an opportunity to assess progress toward accomplishing long-term outcomes, as well as the potential barriers or enablers influencing achievement of long-term outcomes after the program ends.

From Collective Creation to Collective Analysis: Insights for Practitioners

Enterprise evaluation is a practical approach to measure collective outcomes across multiple projects. Through the 3 stages of collective creation, individual collection, and collective analysis, enterprise evaluation provides an opportunity to assess the impact of jointly funded projects in a holistic and multiscalar manner that draws on both collective assessment and individual programmatic approaches to evaluation. In GRHOP, the collective creation of the enterprise logic model facilitated the partners' focus on the collective outcomes and clearly articulated the ways each project feeds into the larger whole. Individual data collection maximized efficiency by allowing projects to collect data independently. Since individual data collection was informed by the enterprise logic model, this approach also ensured that the data collected related to collective outcomes. Individual data collection was also essential, given the broad scope of project objectives and diversity within and across states and clinic systems. Early analysis of individual project data indicates areas of progress toward collective outcomes. GRHOP's experience with enterprise evaluation resulted in several lessons learned and insights for other projects considering a similar approach. One is the need to streamline evaluation efforts with existing data systems. To limit the burden on clinic and community partners, some projects aligned data collection efforts with partners' existing data systems. For example, PCCP made significant changes to its data collection approach to reduce the burden on participating agencies, shifting away from a unique survey and instead relying on the existing Uniform Data System (UDS) measures that the Health Resources and Services Administration already required clinic systems to collect and report. While this approach was favorable for institutional partnerships, a trade-off resulted since PCCP was no longer able to access data disaggregated by clinic site because UDS measures are reported at the operator level. Similarly, MBHCP evaluation data collected by project staff varied substantially from data extracted from electronic health records, given differences in technological capacity at clinics, resulting in inconsistencies in the data available across sites.

Like most jointly funded public health projects, GRHOP's collective evaluation efforts were not mandated and central coordination of evaluation was not funded. The formation of an evaluation subcommittee to oversee evaluation efforts, however, provided accountability and ensured timeliness in individual data collection. Subcommittee meetings allowed partners to consistently share information about evaluation strategies and seek ways of maximizing programming to meet clinic and community needs. However, we recommend that future groups mandate and fund enterprise evaluation efforts so that team members have the resources to take a more proactive role in the coordination of individual data collection, including the development of a centralized action plan for each project's data collection, evaluation deadlines and milestones, and the identification of a team member with evaluation expertise who would be accountable for each evaluation component. Funding specifically for evaluation staffing and coordination should also be included in project budgets. Similarly, it would have been beneficial to have even more dedicated time for data collectors to interact and more frequent meetings of the evaluation subcommittee. These interactions could have allowed team members to provide more frequent updates, resolve data collection challenges, and collectively reflect on progress and outcomes throughout implementation. We also recommend creating the enterprise logic model earlier in the process. In GRHOP's case, the model was not finalized until after project implementation and project-specific evaluation efforts began due to the lack of mandate for evaluation and the urgency for projects to deliver services as quickly as possible. However, the built-in flexibility of the enterprise evaluation approach also allows for its potential application to interrelated projects that have already carried out independent evaluations. For example, projects with similar goals, dedicated team members, and available resources for evaluation could retrospectively develop a collective logic model and draw from their existing evaluation data to engage in collective analysis.

GRHOP's experience with the enterprise evaluation approach represents a fundamental shift in evaluation and academic collaboration as it actively encouraged project partners to deepen their awareness of and collaboration with other projects to maximize impact. In this regard, shared values and strong leadership were key enabling factors for the program's success with the enterprise evaluation approach. Since evaluation was not part of the program's official mandate and partners were responsible for funding their own evaluation efforts, the projects were limited in the amount of time and resources available for evaluation. It was thus critical that partners personally prioritized assessing collective impact and conducting rigorous evaluation, particularly given the complexity of the program's design and the challenges it posed for collective evaluation. As previously mentioned, GRHOP projects involved broad objectives, disciplines, and target audiences. There was a large number and variety of institutional partners and each project had distinct pacing and timing of implementation. The cumulative impact of projects also differed by site and state since projects overlapped at some sites, geographic areas, and communities, but not in all cases. Funding levels also varied by state and project. Considering this complexity, the collective creation phase of the enterprise evaluation approach significantly benefited from facilitation by an external consultant who guided the development of the enterprise logic model. In addition, there were several “champions” for the enterprise evaluation approach among partners who spearheaded the framework's implementation. Several project leaders also had considerable assessment and evaluation expertise, as well as strong preexisting ties to communities. Leveraging existing expertise was highly valuable in facilitating the process.

Conclusion

The enterprise evaluation approach addresses a long-standing policy gap in public health by incorporating cross-sector coordination and collective impact measures into evaluation. The built-in flexibility of the framework allowed GRHOP's individual projects to tailor data collection to context, objectives, and indicators of interest, while also examining the bigger picture of how projects interact and influence collective outcomes. Implementation of the framework takes intentional effort and collaboration, but ultimately it promotes the type of evaluation needed to assess GRHOP's sustainable long-term impact across the Gulf Coast. For funding agencies, enterprise evaluation presents an important opportunity to move evaluation beyond descriptive measures aimed at accountability and gain meaningful insights into the collective impact of interrelated public health initiatives, while still considering individual program impacts. Broader implementation of the framework, however, would require funding agencies to shift their orientation to evaluation. This implies changing the way requests for proposals are written and ultimately altering the very way public health research and capacity building are conceptualized and implemented.

Enterprise evaluation presents an opportunity for public health agencies to shift their evaluation approach to gain better insight into the collective impact of funded projects.

Enterprise evaluation seeks to maximize collective impact by fostering awareness and collaboration among interrelated projects.

The enterprise evaluation framework counters standard public health policies and practices by deliberately incorporating cross-project coordination and collective impact measures.

Measuring collective impact across projects requires intentional effort and strong partnerships among practitioners and institutions.