Can a collective-impact initiative improve well-being in three US communities? Findings from a prospective repeated cross-sectional study.

INTRODUCTION: Communities are seeking to learn if and how they can improve the well-being of their residents. We therefore examined the impact of a community-led, collective-impact initiative, deployed through Blue Zones Project by Sharecare, aimed at improving health and well-being in one set of US communities.
METHODS: We used data from cross-sectional surveys of the Well-Being Index (2010-2017) to assess how the Life Evaluation Index (LEI) in Hermosa Beach, Manhattan Beach and Redondo Beach in California (Beach Cities) changed over time and how this change compares with change for similar cities (Beach Cities-like) and for the USA as a whole. We examined types of interventions, perceived impacts, and relationships between intervention type and change in LEI.
RESULTS: The Beach Cities experienced greater increases in LEI than Beach Cities-like communities and the nation. The entire portfolio of interventions was positively associated with change in LEI in the Beach Cities (+1.12, p=0.012), with process-oriented interventions most closely associated with improvement.
CONCLUSIONS: Community-led collective action that leverages community engagement and activation, strategic use of programming and large-scale built-environment and policy change can improve health and well-being at scale. © Author(s) (or their employer(s)) 2021. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

We conducted a repeated cross-sectional study using the largest longitudinal dataset on well-being of the US population along with data from targeted oversampling of local communities to examine the impact of a community-led, collective-impact initiative aimed at improving the health and well-being of residents, in one set of US communities over an 8-year period.

We measured well-being using the Life Evaluation Index (LEI), a measure frequently used as a surrogate for well-being and a known predictor of population health outcomes, along with select health behaviours and other health-related outcomes.

We assessed how the LEI of residents of Hermosa Beach, Manhattan Beach and Redondo Beach in California changed over time and how this change compares with change in LEI both for similar cities in the USA and for the USA as a whole.

We examined the types of interventions (process, programmatic, policy), their perceived impacts and the relationships between type of intervention and change in well-being as measured by LEI.

While the outcomes are subjective self-reported outcomes, leaving the potential for responses to change over time in a way that is unrelated to underlying life evaluation, the measure employed, Cantril Self-Anchoring Scale, has been thoroughly tested for reliability and validity.

Introduction

Well-being is a holistic, positively framed conception of health that integrates physical, emotional, social, financial, community and spiritual aspects. High well-being is an outcome desirable to be achieved for its own sake1–4 and because it is associated with other valued outcomes, including longer life expectancy, lower healthcare costs and utilisation and lower rates of preterm birth.5–10 As such, communities are seeking to learn if and how they can improve the well-being of their residents.11–16

A number of community factors are associated with higher population well-being, including factors from the social and economic, clinical care and physical environment sectors.17 While evidence from interventional studies to improve well-being at the community level is limited, it likely requires strategic and coordinated partnerships across these sectors. In 2011, Kania and Kramer defined collective impact as the commitment of a group of important actors from different sectors to a common agenda for solving a specific social problem at scale rather than from the isolated intervention of individual organisations.18 The goal of such initiatives is large-scale change that results from broad, cross-sector, coordinated efforts that meet five conditions to produce alignment and lead to results: common agenda, shared measurement, mutually reinforcing activities, continuous communication and backbone support organisations. Evidence of the effectiveness of this approach is still limited, but examples suggest greater progress could be made in alleviating many serious and complex problems if non-profits, governments, businesses and the public were brought together around a common agenda to create collective impact.18 19 We hypothesised that a collective impact approach could be used to increase population well-being.

We examined the impact of a set of coordinated physical-environment and social-environment interventions deployed through the Blue Zones Project (BZP) by Sharecare, a community-led, collective-impact initiative aimed at improving the health and well-being of residents, in one set of US communities over an 8-year period.20 This set of US communities chose a collective impact approach because they wanted to measurably improve well-being across their entire population by changing policies, systems and environments to achieve greater effect across the population and by leveraging the relationships between organisations to achieve shared objectives and outcomes. Our primary outcome was the Life Evaluation Index (LEI), a self-reported measure of one type of subjective well-being that is commonly used in aggregate as an indicator of population well-being and is established as a known predictor of population health outcomes.21 22 We assessed how the LEI of residents of Hermosa Beach, Manhattan Beach and Redondo Beach in California (Beach Cities) changed over time and how this change compares with change in LEI both for similar cities in the USA and for the USA as a whole. We also assessed how health behaviours and outcomes changed over time for Beach Cities’ residents. Finally, we examined the types of interventions, their perceived impacts and the relationships between type of intervention and change in well-being as measured by LEI. We also discuss lessons learnt from the community-based team that has led the initiative and present methods for assessing a collective-impact intervention to improve population well-being.

Methods

Conceptual framework: implementing a collective impact approach

Inspired by the world’s longest-lived societies, BZP is a community-led, well-being improvement initiative that strives to improve the environments of people to support sustainable, healthy behaviour change and subsequently reduce healthcare costs, improve productivity and boost economic vitality of communities. Since 2010, over 55 communities across North America have launched BZP. For the BZP launched in the Beach Cities in 2010, the Beach Cities Health District (BCHD) served as the backbone organisation to support and advance evidence-based, scaleable programme implementation, sector collaboration and community mobilisation. BCHD is a preventive-health agency serving Hermosa Beach, Manhattan Beach and Redondo Beach, California, since 1955.23

To achieve a common agenda, BCHD elevated health and well-being across the three cities, using data to create a common understanding. The effort resulted in a BZP blueprint that included overarching vision, goals, strategies and metrics used across sectors and workgroups. For their shared measurement system, BZP used the annual Well-Being Index (WBI) survey to collect, analyse and report valid, reliable data over time. The BZP also established a set of benchmarks for the Beach Cities to achieve to become a certified Blue Zones Community. To accomplish mutually reinforcing activities, BZP used tactics such as an annual blueprint, evidence-informed sector pledges and workgroups to coordinate activities in alignment with the blueprint. To implement continuous communication, BZP communicated frequently with community leaders and committee members and the public through media partnerships. BCHD serves as the backbone support organisation, galvanising and championing BZP, providing governance and decision-making, acting as convenor for population health and well-being and sustaining its commitment to this initiative. For more details on how BZP implemented the collective impact framework, see the online supplemental file 1. Through BZP activities over 8 years, BCHD engaged >26 000 residents, activated 160 employers to create healthier work environments, supported 120 BZP-approved restaurants and grocery stores to make healthy changes to the food environment, engaged policy-makers to adopt built-environment and tobacco-control policies and partnered with the school districts to integrate gardens, physical activity and social-emotional wellness into schools.

Study population

Our study population included residents who completed the WBI survey from 2010 to 2017 in Hermosa Beach (2010 Census population: 19 422), Manhattan Beach (population: 34 986) and Redondo Beach (population: 75 043). Cross-sectional surveys were conducted in 2010, 2011, 2012, 2013, 2015 and 2017.

Data

We used multiple sources of data, including the national WBI, the targeted WBI in the Beach Cities described above and BCHD staff themselves. We used data from the WBI from January 2010 through December 2017.24 The WBI survey instrument itself has been revised several times. To focus on trends over time, we restricted our analyses to items that were administered over all 8 years. More information about the development of the WBI is included in the online supplemental file 1. From 2008 through 2017, Gallup and Sharecare interviewed approximately 500–1000 adults across the USA nearly every day, for a total of approximately 160 000–353 000 interviewed US adults annually. Concurrently, Gallup and Sharecare conducted a targeted survey in which they achieved a representative sample for the three cities that comprise the Beach Cities in the autumn of the above years.

Primary outcome

Our primary outcome was the aggregate LEI score. The LEI is measured using The Cantril Self-Anchoring Scale,25 which consists of the following prompt and questions: Please imagine a ladder with steps numbered from zero at the bottom to 10 at the top. The top of the ladder represents the best possible life for you and the bottom of the ladder represents the worst possible life for you. The following questions are asked: (1) On which step of the ladder would you say you personally feel you stand at this time? (2) On which step do you think you will stand about 5 years from now? The first item measures current life satisfaction (CLS) and the second measures future life optimism (FLO). Based on statistical studies of CLS scores, FLO scores and their relationships with other items, Gallup categorises respondents into three groups: thriving, struggling and suffering.26 Respondents with positive views of their present life situation (CLS 7–10) and positive views of the next 5 years (FLO 8–10) are categorised as thriving. Respondents with negative views of their current life situation (CLS 0–4) and negative views of their future life (FLO 0–4) are categorised as suffering. All other respondents are categorised as struggling. The LEI, calculated as [mean(thriving)–mean(suffering)]×100, was calculated for each zip code. We also analysed individual CLS and FLO scores.

Secondary outcomes

Secondary outcomes included self-reported behaviours (fruit and vegetable consumption, weekly physical activity, smoking), healthcare access characteristics (uninsured status, having a personal medical doctor), health outcomes (overweight, obesity) and emotional health indicators (stress, worry or enjoyment yesterday). Each variable is assessed with a single self-report item in the WBI. Overweight and obesity are calculated based on self-reported height and weight.

Independent variables

Independent variables were community-based interventions that changed the physical and social environment in the Beach Cities. To characterise changes made by BZP, we used community-reported data on the perception of changes in the community as well as the number and strength of each implemented intervention as assessed by the BCHD staff.

The data on awareness and perception of physical-environment and social-environment community interventions were obtained from the targeted WBI survey of the Beach Cities residents that asked participants items about their awareness of BZP, engagement in the project, perception of its impact and perception of the current (2017) built environment and change in built environment from 2010 to 2017. Engagement was measured through self-reported assessment in the WBI survey and by tracking community member and organisation participation in BZP interventions. BZP evaluated the demographic characteristics of those engaged to understand those who were and were not being reached. BCHD then made efforts to respond to populations with lower engagement, such as residents aged 18–25 years.

BCHD staff compiled a database of physical-environment and social-environment interventions by year in Beach Cities, from 2010 through 2017. These interventions are listed and categorised as process-focused, program-focused or policy-focused interventions in table 1. All interventions were attributed equally to all three communities, with start and end dates used to attribute each intervention to one or more calendar years. Process interventions focused on establishing infrastructure to create and sustain the initiative, creating buy-in from stakeholders, building community coalitions and capacity and creating communication strategies (eg, establishing community workgroups, identifying champions across sectors, creating communications strategies). Programme interventions involved delivery of specific programmes to different audiences with the intention of bringing people together to build supportive social networks, teach skills that support health-promoting behaviour change and provide access to the larger initiative (eg, walking groups, plant-based potluck groups, purpose workshops, worksite wellness activities, volunteer opportunities). Policy interventions included city or school-district level policy change aimed at improving the built environment or healthy behaviours (eg, construction of pedestrian and bicycle facilities, adoption of tobacco ordinances, changes in food environment).

Table 1

Process-focused, program-focused and policy-focused interventions and their annual impact scores in the Beach Cities, 2010–2017

Intervention	2010	2011	2012	2013	2014	2015	2016	2017
Process-focused	0.000	1.962	3.513	4.907	4.716	4.276	3.198	3.086
WBI data collection	0.000	0.055	0.055	0.055	0.000	0.055	0.000	0.055
Build leadership team	0.000	0.303	0.303	0.303	0.166	0.166	0.166	0.000
BZP promotion	0.000	0.000	0.550	0.550	0.550	0.055	0.030	0.030
BZ community certification	0.000	0.000	1.000	1.000	1.000	1.000	0.001	0.001
BZP programme structure	0.000	0.303	0.303	1.000	1.000	1.000	1.000	1.000
BZP volunteer programme structure	0.000	0.303	0.303	1.000	1.000	1.000	1.000	1.000
WBI data reporting	0.000	1.000	1.000	1.000	1.000	1.000	1.000	1.000
Program-focused	0.000	0.479	0.524	1.521	1.422	1.422	2.149	2.124
Purpose events	0.000	0.030	0.030	0.303	0.166	0.166	0.030	0.006
Worksite wellness	0.000	0.030	0.030	0.550	0.001	0.001	0.100	0.100
Alliance for a healthier generation	0.000	0.010	0.055	0.100	0.100	0.100	0.550	0.550
Walking school bus	0.000	0.100	0.100	0.100	0.550	0.550	0.166	0.166
Moais	0.000	0.303	0.303	0.303	0.303	0.303	0.303	0.303
MindUp	0.000	0.006	0.006	0.166	0.303	0.303	1.000	1.000
Policy-focused	0.000	0.410	0.890	1.410	1.788	3.365	3.410	3.410
SCAG Aviation Boulevard grant	0.000	0.000	0.000	0.000	0.030	0.055	0.100	0.100
BZ grocery designation	0.000	0.000	0.100	0.100	0.100	0.100	0.100	0.100
Beach Cities livability plan	0.000	0.100	0.100	0.100	0.100	0.100	0.100	0.100
South Bay bicycle master plan	0.000	0.100	0.100	0.100	0.100	0.100	0.100	0.100
City policy workshops	0.000	0.055	0.550	0.550	0.303	0.010	0.010	0.010
Redondo Beach improvements	0.000	0.100	0.010	0.010	0.055	1.000	1.000	1.000
Tobacco control ordinances	0.000	0.000	0.000	0.000	0.550	1.000	1.000	1.000
BZ restaurant designation	0.000	0.055	0.030	0.550	0.550	1.000	1.000	1.000

BZP, Blue Zones Project; SCAG, Southern California Association of Governments; WBI, Well-Being Index.

Following an approach adapted from the Healthy Communities Study Team,27 BCHD staff independently assessed each intervention on three dimensions: strategy, reach and duration. For each intervention, each dimension was rated as high, medium, low or none for each calendar year. For strategy, interventions rated high were those that modified policies and systems, changed consequences, modified access, opportunities and barriers or changed infrastructure; those rated medium enhanced services and supported interventions and those rated low provided information and enhanced skills. For reach, interventions rated high impacted the whole community or had system-wide impacts; those rated medium reached certain communities; those rated low reached small groups or committees. For duration, interventions rated high were ongoing or occurring multiple times in a year; those rated medium occurred once a year and those rated low were categorised as one-time events. Differences between raters were discussed and resolved by consensus. For each intervention and each calendar year, a score of 1.0, 0.55, 0.1 or 0 was assigned to each of the three dimensions based on ratings from high to none, respectively. An annual impact score was assigned to each intervention as the product of the three dimension scores.

Analyses

We first described trends in LEI over time for the Beach Cities. We then compared change over time in LEI for residents of the Beach Cities with change over time in LEI for two comparison groups: Beach Cities-like communities and the nation as a whole. A set of 15 Beach Cities-like communities were chosen using Mahalanobis matching with the Beach Cities communities on income, total population, age distribution, race distribution, employment, educational attainment, initial life evaluation and initial WBI score (online supplemental table 1). Matching on these characteristics identified communities that were demographically similar to the Beach Cities at the start of BZP. We used multilevel mixed-effects linear regression to compare the differences in LEI over time between the Beach Cities and both the Beach Cities-like communities and the USA. Each model included calendar year, an indicator for Beach Cities residence and a random effect for zip code.

Subsequently, we assessed how life evaluation for residents of the Beach Cities differed based on their awareness of and engagement in BZP, their perception of the project’s impact and their perception of the 2017 built environment and change in the built environment from 2010 to 2017. We assessed how self-reported behaviours, healthcare-related characteristics, health outcomes and emotional-health indicators changed over time for Beach Cities’ residents. We summarised their individual LEI scores according to responses to these survey items, and used multilevel mixed-effects logistic regression to compare the differences in these secondary outcomes between the Beach Cities and both Beach Cities-like communities and the USA over time.

Finally, we assessed the association between impact scores of the different intervention categories—process, programme and policy—and life evaluation over time, using mixed-effects linear models where LEI was the dependent variable. Each model included the three impact scores and random effects for time and zip code; in secondary analyses, we introduced a 1-year lag between the interventions and LEI scores.

Patient and public involvement

This research was done with involvement of the community-based team from inception through publication. The community team was involved in all aspects of study design, interpretation of results and the writing and editing of this document.

All data are retrospective and deidentified. All analyses were performed using Stata V.16 (StataCorp, College Station, Texas, USA).

Results

From 2010 through 2017, between 57 and 1460 residents of the Beach Cities communities participated in the annual nationwide WBI survey and between 996 and 1305 participated in each targeted survey. Over these years, between 57 and 155 residents of the Beach Cities-like communities participated in the annual nationwide WBI survey, while the total US sample included 160 498–353 563 participants each year.

Life evaluation

Life evaluation (LEI) in the Beach Cities improved between 2010 and 2017 (p<0.001) (table 2). Over these 8 years, CLS improved in the Beach Cities (p=0.038), while FLO remained unchanged (p=0.445). Greater improvements in life evaluation were also experienced in the Beach Cities when compared with the Beach Cities-like communities (p<0.001), again driven primarily by greater gains in CLS (p=0.002). Although life evaluation improved for the nation from 2010 to 2017, the Beach Cities experienced greater improvement than the nation for LEI (p<0.001), CLS (p<0.001) and FLO (p<0.001). These trends are visualised in figure 1.

Table 2

Change over time in Life Evaluation Index, current life satisfaction and future life optimism in Beach Cities versus Beach Cities-like communities and the nation, 2010–2017

	Life Evaluation Index	P value	Current life satisfaction	P value	Future life optimism	P value
Beach Cities versus Beach Cities-like communities
Beach Cities-like communities	(ref)		(ref)		(ref)
Beach Cities	6.64	<0.001	0.36	0.002	0.22	0.057
Beach Cities # elapsed year since 2010	−0.21	<0.001	−0.02	0.240	−0.02	0.222
Beach Cities versus nation
Nation	(ref)		(ref)		(ref)
Beach Cities	15.60	<0.001	0.50	<0.001	0.55	<0.001
Beach Cities # elapsed year since 2010	−0.31	<0.001	0.00	0.651	−0.03	0.020

Figure 1

Impact of policy, process and programme interventions in the Beach Cities by year, 2011–2017.

Awareness, engagement and perception of change in the built environment over time

In the Beach Cities in 2017, 37% of residents reported being aware of BZP. Of those who were aware, 44% reported being either somewhat or highly engaged in BZP, 34% stated that BZP had a positive impact on their individual lives and 62% agreed that BZP had a positive impact on the community. The residents who reported in 2017 that they were aware of BZP had a higher mean LEI score than residents who reported that they were not aware (p=0.006). Residents with higher levels of engagement also had higher mean LEI scores than residents with lower levels of engagement (p=0.034). Those who described greater ease in moving about by biking or walking since 2010 and those who described greater ease in moving about by public transportation since 2010 also had higher mean LEI scores than those who described greater difficulty in those two aspects of community life (table 3).

Table 3

Beach Cities’ resident awareness, participation and perception of impact of Blue Zones Project and associations with life evaluation in Beach Cities, 2017

	N	(Weighted %)	Life evaluation score
Awareness of Blue Zones Project in the Beach Cities			0.006
Yes	482	(36.87%)	79.51
No	710	(63.13%)	76.93
Level of engagement in Blue Zones Project			0.034
Highly engaged	28	(5.67%)	81.96
Somewhat engaged	209	(38.30%)	81.17
Not at all engaged	244	(56.03%)	77.86
Positive impact of Blue Zones Project on one’s life			0.149
Strongly agree	71	(15.00%)	82.32
Agree	96	(19.40%)	81.56
Neither agree nor disagree	149	(34.01%)	78.59
Disagree	57	(10.98%)	78.86
Strongly disagree	104	(20.62%)	77.86
Positive impact of Blue Zones Project on community			0.042
Strongly agree	159	(34.97%)	81.92
Agree	124	(27.34%)	79.68
Neither agree nor disagree	121	(26.70%)	76.98
Disagree	29	(6.35%)	77.07
Strongly disagree	31	(4.64%)	82.10
Ease of moving about by biking or walking			<0.001
Very easy	645	(53.97%)	79.64
Moderately easy	393	(33.35%)	77.64
Moderately difficult	92	(8.09%)	77.39
Difficult	61	(4.60%)	63.11
Ease of moving about by public transportation			0.026
Very easy	172	(16.44%)	81.28
Moderately easy	414	(34.25%)	77.50
Moderately difficult	291	(25.89%)	77.85
Difficult	245	(23.42%)	76.75
Ease of biking or walking compared with 2010			<0.001
Much easier	74	(7.25%)	78.92
Easier	138	(11.96%)	80.51
About the same	802	(71.87%)	78.53
More difficult	98	(6.47%)	71.68
Much more difficult	34	(2.46%)	71.62
Ease of using public transportation compared with 2010			<0.001
Much easier	47	(5.18%)	84.15
Easier	138	(11.69%)	77.64
About the same	789	(75.54%)	78.60
More difficult	67	(5.29%)	70.52
Much more difficult	29	(2.30%)	72.24
Eating unhealthy foods to deal with stress			0.022
Never	167	(13.11%)	78.83
Rarely	406	(34.69%)	79.42
Occasionally	503	(40.95%)	77.18
Most days	96	(9.47%)	74.43
Every day	25	(1.79%)	74.00
Having difficulty sleeping			<0.001
Never	182	(13.81%)	81.77
Rarely	408	(34.56%)	79.04
Occasionally	377	(31.18%)	77.48
Most days	165	(15.10%)	75.64
Every day	68	(5.34%)	68.90
Smoking or drinking alcohol to deal with stress			0.136
Never	709	(56.20%)	78.68
Rarely	265	(23.32%)	77.26
Occasionally	167	(15.95%)	77.01
Most days	31	(2.67%)	76.13
Every day	25	(1.86%)	71.60
Wanting to live and age in one’s current community			<0.001
Strongly agree	632	(43.01%)	79.90
Agree	281	(27.23%)	76.90
Neither agree nor disagree	155	(15.81%)	75.61
Disagree	68	(8.69%)	74.63
Strongly disagree	58	(5.25%)	72.41

Secondary outcomes

Over the course of the study, the proportion of people who reported having a personal doctor in the Beach Cities increased by 1.5% more per year than in the Beach Cities-like communities (95% CI 1.1 to 2.0; p<0.001) and increased by 2.3% more per year than the nation (95% CI 1.9 to 2.5; p<0.001). At the same time, the proportion of people who reported high weekly exercise decreased by 1.4% (95% CI −2.4 to −0.4; p<0.001) compared with the Beach Cities-like communities, although this was unchanged compared with the nation over time. Finally, the proportions of people who reported stress and enjoyment were unchanged relative to Beach Cities-like communities, but decreased by 0.9% (95% CI −1.4 to −0.4; p<0.001) and increased by 0.4% (95% CI 0.0 to 0.8; p=0.022), respectively, relative to the nation. All other changes in secondary outcomes over time did not differ significantly compared with the Beach Cities-like communities and national comparison groups.

Impact of interventions

Interventions and their annual impact scores are detailed in table 1. Overall impact for each intervention by year is visualised in figure 1. Across all years, BCHD staff perceived greatest impact from their process-focused interventions (eg, building leadership team, building and maintaining capacity for promotion of the initiative, creating certification standards to guide organisations to create healthier environments and collecting and reporting WBI data to raise awareness and educate the public). Policy-focused interventions received the second highest cumulative impact scores. These interventions included major built-environment changes (eg, creating a protected bike path) and local policy adoption (eg, passage of tobacco ordinances, Complete Streets policies, regional bicycle plan, capacity-building workshops). Programmatic interventions (eg, starting and scaling fitness, wellness and positive psychology programmes in schools, workplaces and the community-at-large) received the lowest cumulative impact scores.

The entire portfolio of interventions was positively associated with change in LEI over time in the Beach Cities (+1.12, p=0.012). As a whole, process-focused interventions were positively correlated with change in LEI (+2.88, p<0.001), whereas policy-focused interventions were negatively associated with change in LEI (−3.28, p<0.001). Programmatic interventions were comparatively neutral to change in LEI (−0.09, p=0.714). In analyses in which impact scores from the prior year were used to predict current year LEI, effects were as follows: process-focused (0.86, p<0.001), policy-focused (−1.02, p<0.001) and programmatic interventions (−1.99, p<0.001).

Discussion

We evaluated if and how the strategic implementation of a community-led, collective-impact initiative influenced the well-being of residents in three communities (known as the Beach Cities) in California. Our results showed that such an initiative, sustained over multiple years, can contribute to significant improvements in health and well-being. From 2010 to 2017, the Beach Cities experienced greater increases in overall life evaluation than experienced by both the Beach Cities-like communities and the nation. The difference between the Beach Cities and the Beach Cities-like communities was driven largely by greater increases in current life satisfaction in the Beach Cities; whereas, the difference between the Beach Cities and the nation was driven by greater increases in both current life satisfaction and future life optimism in the Beach Cities. In addition to overall life evaluation, the Beach Cities outpaced the two comparison populations in other outcomes as well, particularly higher percentages of people reporting having a personal doctor in the Beach Cities than in both Beach-like communities and the nation as a whole as well as lower levels of stress and higher levels of enjoyment in the Beach Cities than in the nation.

When studied altogether, the entire set of interventions of the collective-impact initiative was positively associated with the increase in life evaluation experienced in the Beach Cities. When we assessed the various elements of the initiative, we found the process-oriented interventions to be the most closely associated with improvements in life evaluation. These elements, consistent with the ‘early changes’ phase described in the literature on collective impact,19 lay the foundation and support for successful systems and policy change. These elements included efforts to create and strengthen partnerships, foster political will, create coalitions, develop essential infrastructure and capacity, foster community leadership, develop the framework and systems to use data and define shared goals and objectives. These results affirmed the qualitative assessment from staff who saw their early steps to bring together local leadership to identify shared goals, objectives and strategies and their sustained investment in capacity building, community engagement and transparent measurement as essential to the success of the initiative. These results also affirm the importance and value of these elements of the collective-impact model.

Our results show how the programmatic elements of the Beach Cities initiative did not contribute independently to the observed population improvements in well-being. This finding resonates with the perspective of the BCHD staff who observed that it was the process-oriented step of coming together to collaborate with a sense of urgency towards a defined and shared outcome that mattered most. The process interventions were typically more foundational and broader in scope than the more well-defined or tightly scoped programme interventions. Because larger, more sustained elements tended to be assessed as having greater impact, process interventions that were sustained over time scored higher than programme interventions. Non-sustained interventions as well as various smaller-scale programming may have played an important function in the coming together to collaborate, although they were insufficient on their own to create broad, community-level change. The BCHD staff acknowledged the limitations in this assessment approach while also affirming the value of fostering a ‘movement mentality’. As Lauren Nakano, one of the members of the BCHD team, reflected, “Shifting our focus at BCHD to policies, systems, and environment offered a new strategy for addressing community health in the Beach Cities. What we have learned is combining Collective Impact process-oriented interventions with programmatic interventions yield greater outcomes with longer-term impacts”.

In our analyses, policy-focused interventions were unexpectedly inversely correlated with the LEI, when assessed both concurrently and with a 1-year time lag, although the magnitude of the effect was smaller in the model with the 1-year time lag than in the model without lag. One possible explanation for this finding is that policy change and large-scale built environment change can meet resistance and be difficult to both enact and implement.28 29 Initial challenges in large-scale change may have resulted in perceived negative impacts at first. With time, as residents experience the benefits of the policy or built-environment change, community sentiment may change and the positive impacts on health and well-being may become evident and accumulate. Also, as noted in When Collective Impact has an Impact, collective impact is a long-term proposition with sites showing population change in initiatives in operation from 8 to 25 years, with an average of 14 years, placing priority on laying a strong foundation.19 To assess whether policy interventions require more time to positively impact population well-being, we would require data over a longer time period.

Meaningfully assessing the impact of a collective-impact initiative brings considerable challenges as well as key opportunities. Challenges arise from the number and variety of interventions included in such an effort, the number and diversity of populations reached via different interventions, the scale of the desired impact and the timeline required for impact at scale.30 To address these challenges, we adapted a methodology for assessing impact of complex community-based interventions, systematically harnessing on-the-ground knowledge of interventions and their perceived impacts.27 Longitudinal tracking of outcomes that matter is necessary for this kind of evaluation, to assess overall impact and to support evaluation of potential mechanisms to improve well-being. This study was possible as a result of a concerted longitudinal effort to collect person-centred measures of well-being, among other measures. Moreover, as expected with complex, long-term endeavours, BCHD experienced multiple challenges in implementing a collective impact framework, including challenges in (1) recruiting and sustaining community member and leadership participation, (2) cultivating leaders with system leadership skills and managing turnover of highly instrumental stakeholders (eg, elected officiials, CEOs, key staff), (3) maintaining focus on system strategies, (4) implementing and sustaining multisector interventions, (5) managing communication to sustain awareness, interest and involvement and (6) managing the evolution of programmes and campaigns to keep them ‘fresh’ while maintaining fidelity.

This study has several limitations. First, given the relative affluence and homogeneity of the residents of the Beach Cities and a starting average life evaluation higher than the national average at the time, there may be concerns regarding the generalisability of our findings. Despite starting at a relatively high average life evaluation, the Beach Cities experienced even greater improvements in overall life evaluation, as well as in other outcomes, than demographically similar cities and the nation as a whole. Moreover, while specific interventions may vary between communities, this study shows how coordinated interventions aimed at sustainable change can contribute to the collective well-being of a community and be assessed in a systematic way. Second, the targeted WBI survey in the Beach Cities was conducted in autumn of each collection year, whereas survey data for the comparison populations were collected throughout the entire year, thereby introducing the concern that seasonal differences in well-being may be differentially present in the different samples. However, maintaining the same survey window for the Beach Cities-specific survey allows for comparison across years without concern that seasonal effects may be introducing bias into the observed changes over time. Third, the primary outcome measure used in this study measures one type of subjective well-being. Although widely used as an indicator of population well-being,25 31 aggregated life evaluation does not provide a complete assessment of community well-being. Fourth, as with any survey, response bias could threaten the representativeness of the data. Sampling and weighting methods were applied to manage response bias and produce data representative of the populations surveyed. Fifth, as captured by the ‘plausible rival explanations’,19 other efforts and trends external to BZP could have contributed to the observed improvements in health and well-being. Recognising this potential limitation, we intentionally included individuals who were on-the-ground in the Beach Cities for the years of the initiative on the study team and conducted analyses comparing change in the Beach Cities with change both in comparable cities and the nation as a whole.

Conclusion

A group of US communities united by a shared goal of greater well-being has demonstrated that a community-led, collective-impact initiative can achieve greater health and well-being for its residents. Process-oriented elements such as building will and leveraging social capital, developing capacity, creating coalitions, fostering community leadership and defining shared goals and objectives were most closely associated with improvements in population well-being as measured by the LEI. Community-led collective action that leverages all intervention types from community engagement and activation, strategic use of programming and large-scale built-environment and policy change together can improve health and well-being at scale.