Does Implementation Follow Design? A Case Study of a Workplace Health Promotion Program Using the 4-S Program Design and the PIPE Impact Metric Evaluation Models.

OBJECTIVE: The aim of this study was to describe the content of a multiyear market-based workplace health promotion (WHP) program and to evaluate design and implementation processes in a real-world setting.
METHODS: Data was collected from the databases of the employer and the service provider. It was classified using the 4-S (Size, Scope, Scalability, and Sustainability) and PIPE Impact Metric (Penetration, Implementation) models. Data analysis utilized both qualitative and quantitative methods.
RESULTS: Program design covered well the evidence-informed best practices except for clear path toward sustainability, cooperation with occupational health care, and support from middle-management supervisors. The penetration rate among participants was high (99%) and majority (81%) of services were implemented as designed.
CONCLUSION: Study findings indicate that WHP market would benefit the use of evidence-based design principles and tendentious decisions to anticipate a long-term implementation process already during the planning phase.

BACKGROUND

Employers are increasingly interested in providing their employees access to health promotion programs,[1] hoping to capture return of investment (ROI).[1,2] Systematic reviews, long-term studies, and meta-analytic findings indicate that workplace health promotion (WHP) programs can improve health and generate savings.[2-6]

However, WHP is a broad concept with considerable heterogeneity in terms of settings and program characteristics, and evidence regarding the impact of different programs is still limited and inconsistent.[4,6-9] Furthermore, new services and concepts are constantly emerging. This is problematic from a WHP evaluation perspective, because an upcoming concept is seldom put into rigorous scientific testing such as randomized controlled trials. It has been suggested that in order to pilot concept in a real-world setting, evaluation tools should be feasible, valuable information should be produced without a control group, and feedback loops should be offered for quality improvement.[9-11]

Generally, WHP programs are expected to achieve positive outcomes.[9] These outcomes can be evaluated according to improvements in four broad categories: health and well-being of employees; cost savings; individual and business performance; and human capital.[4,9] To clarify the expectations, Goetzel et al (2014, p. 928) noted: “It is unlikely that an employer would spend a few hundred dollars per employee…and achieve all of these results.”[9] Pronk[12] posited that whereas substantial financial savings may come from disease management programs targeted at poor health status employees, additional value is generated across the entire population even when portions of this value are difficult or impossible to monetize. This value may be difficult to measure financially in the worksite setting, which, despite available evidence, is recognized as a complex and challenging environment in which to generate impact.[13,14]

To contribute successful programming, health professionals and researchers have given recommendations for evidence-based practices in WHP. These include, for example, combining health risk assessment (HRA) to feedback and targeted services, paying attention to individual's readiness to change, giving social support, offering wide appeal of services with an easy accessibility, senior management involvement, and supporting self-care and self-management.[15-18] Recently, Pronk[19] compiled a comprehensive summary list of 44 best practices. The practices were categorized into nine larger dimensions, including leadership, relevance, partnership, comprehensiveness, implementation, engagement, communications, being data-driven, and compliance.[19] The best practices were subsequently validated with promising results in the US.[20,21]

Even though the literature offers multiple recommendations on how a WHP program should be designed,[9,12,15-19] there are no investigations on how well a market-based concept meets these recommendations, and how a multiyear implementation process follows planning decisions. The present study focused on examining the planning process behind of a real-world program (hereafter referred to as ENSO). ENSO can be classified as a comprehensive program based on the Healthy People 2010 definition.[22] The ENSO design process is described with a 4-S model and the implementation phase is evaluated with the PIPE Impact Metric model.[23] A detailed background of the models has been described previously.[23] Both models are based on a system approach offering a suitable evaluation tool for a real-world setting.[11,23,24] They were recently used in evaluating diabetes prevention programs.[25,26]

In the present study, “4-S” describes the planned Size of the program's impact on population, Scope of the services, Scalability to reach the intended audience, and the Sustainability resources of the program to keep the process running. Finally, the whole design process of ENSO is contrasted to the before mentioned nine best practice dimensions.[19] The implementation process is analyzed with the PIPE Impact Metric model's Penetration and Implementation components, which reflect the “realized investment” in order to generate expected outcomes. Participation and Effectiveness components that reflect the generated “returns” will be analyzed later as a part of an outcome evaluation study.

METHODS

Study Design

This investigation followed a retrospective quasi-experimental study design, without control group. The study combined quantitative and qualitative data analyses to demonstrate the planning and the implementation processes of ENSO. The design findings were contrasted against nine Best Practice dimensions.[19]

Setting

The ENSO program was formed by three main parties: Stora Enso Metsä was the employer, its employees were the participants, and 4event Ltd was the external provider of health promotion services.[27,28] Stora Enso Metsä is a wood supply organization, which operates through 120 small units, dispersed over Finland. The ENSO program was a tailored version of a complete WHP concept, and it was carried out in Finland by the service provider during January 2010 to March 2014. ENSO was designed by three Finnish wellness professionals, who had at least a Master's education from medicine, nutrition, or physical education. In addition, two HR-executives of the employer were involved in the design process. ENSO was executed in practice by the service provider's head coach and 20 wellness coaches. These coaches all had attained, at a minimum, undergraduate educational degrees in nutrition, physical education, health sciences, or coaching. All three parties complied with the Finnish workplace legislation designed to ensure healthy and safe work environment.[29] The employer's occupational health care units were informed about the program, but they were not involved in ENSO. As a theoretical foundation, ENSO deployed the transtheoretical model (TTM) and a mixture of different behavioral change techniques, such as self-monitoring and motivational interviewing.[30,31] As a benchmark study for ENSO's design, the provider used the study Finnish Health Behaviour and Health among the Finnish Adult Population.[32]

Intervention

The main goal of the program was to improve the health and wellbeing of every employee. The main focus of the services was to support low-effort, pleasant life changes and a positive health conception rather than a negative one. Health problems such as smoking, misuse of alcohol, or depression were not targeted in the content. The provider designed and implemented ENSO by using five main components: HRA, annual worksite health promotion events, targeted services, communication tools, and worksite peer support through playmakers. The playmakers represented the employees and they were selected voluntarily. They were trained by the provider in order to assist in the implementation process of ENSO. The HRA was a launching element of ENSO, and its purpose was to define employees’ health and their needs for targeted services. After the initial HRA in 2010, the provider designed and implemented altogether 27 different services, which were either available to all employees or targeted for specific groups (see Fig. 1). For example, online coaching included a possibility to contact wellness professionals in nutrition, exercise, or work-life balance issues. Targeted phone calls were focused to increase motivation or to check up on musculoskeletal disorders. Annual WHP events took place in groups of 50 to 120 people and offered health education to all employees. The HRA was carried out again during 2013 to 2014. The aims of the intervention as well as the services are described in more detail in the Results section (pages 755–758) as we outline the design phase of ENSO with the 4-S model.

FIGURE 1

Flow chart of the program.

Participants

The ENSO program and HRAs were made available for the whole staff of Stora Enso Metsä (N = 651). Some employees declined the HRAs (10.0% in 2010 to 2011 and 20.9% in 2013 to 2014), but they still had access to the services during the whole study period. External funding or incentives were not used, although the employees participated in the services mostly during paid working hours. When the ENSO program started, there were 651 people working in Stora Enso Metsä. The total number of employees decreased annually from 2010 to 2014 as follows: 651 → 634 → 630 →625 → 530. The largest drop from 2013 to 2014 was due to an establishment of a new organization and statutory negotiations in 2013.

Measurements

The HRA consisted of a questionnaire and nine biometric measurements, which were executed with a transferable Polar BodyAgeTM system[33,34] that includes five physiological factors and four performance factors. The physiological factors were body mass index, body fat percentage, systolic and diastolic blood pressure, VO2max and performance factors the number of crunches in 60 seconds, a leg endurance test, a bicep curl, and a sit and reach test. The HRA questionnaire was a combination of questions from the survey Finnish Health Behaviour and Health among the Finnish Adult Population (exercise, habits), Polar BodyAge test protocol (testing safety), TTM-model (life-change), the employer's own questions (background information), and the provider's own questions (vitality, weight management, musculoskeletal disorders). Altogether, the questionnaire had 36 questions at the baseline.

Data Collection

In this study, the quantitative and qualitative data were collected from different resources (34 pcs) by two representatives of the provider and two representatives of the employer. Each resource was coded and qualitative and qualitative findings were categorized according to the 4-S model in order to be analyzed as a design data by the researcher of this study.[23]

The data for the was collected from the provider's annual program designs and service descriptions containing information (ie, time, mode, duration) of the services. The referred to the range and extent of the program activities.[23] The scope information was gathered from the provider's annual service books, their HRA database, and their orientation materials for coaches. The collected data introduced the target population and the primary content of the planned services. The referred to the ability of a program to systematically follow timed and planned actions (eg, media, registration or communications tools) that increase the reach of a program until a critical mass is attained.[23] In this study, the scalability information was collected from the launching plans of the program in 2010 to 2011 and from the marketing plans in 2011 to 2014. The referred to the needed long-term support to keep the process running.[23] In order to clarify the time use and the organizational support to the program continuum, this study used interviews of the HR-executives, check-up meeting documents, as well as provider's customer resource management (CRM).

To investigate the execution phase of ENSO, data for Penetration and Implementation were assembled.[23] The referred to how well the target population was reached by invitations.[23] The penetration data consisted of both the employer's and the provider's evaluations of the employees, who actually received the invitations. A straight employee evaluation was limited out because of the decrease in the number of employees in 2013 and the data collection begun after 2014. The coefficient represents the degree of realized actions compared with the designed actions.[23] This factor gathered information from the provider's annual budgets compared with the executed actions in CRM database during 2010 to 2014.

Noteworthy, most of the gathered data were provider-centered. To reduce potential bias of the data collection, the data resources and categorized 4-S findings were double-checked by an external WHP professional (PhD, Adjunct Professor in the University of Helsinki). To ensure the employer's aims behind ENSO, two HR-executives of the employer were interviewed in 2015. In these interviews, the background information, that is, HR-strategy alignments, history of the statutory negotiations, and company's own wellness actions were assembled during a timeline 2006 to 2014.

Data Analysis

This study used a mixed-methods approach[35] combining both qualitative and quantitative methods of data analysis. First, the design findings were classified under 4-S model to clarify aims and decisions behind ENSO. Second, the whole design process was compared with the nine Best Practice dimensions.[19] This summarized design analysis was made by the researcher with help of the external WHP professional. Both parties agreed with the results of the comparison. Third, the quantitative analysis was used to describe the selected baseline HRA results and the penetration and implementation rates. To represent the HRA results, a database was exported into SPSS 23 and the frequencies and means of the participants were analyzed. When calculating the penetration rates of ENSO, both the employer's HR manager and provider's representative provided their own penetration estimations. The total penetration rate was counted as an average of these two evaluations. To estimate the implementation coefficient, a check-list of planned-executed and planned-but-not-executed services was formed by the researchers and subsequently adjudicated by the employer, the provider, and the external WHP professional.

Ethical Issues

Due to the retrospective nature of this investigation, all participants and the employer informed consent to participate in the study in the fall of 2013. The protocol of the study was included in the HRA remapping questionnaire in 2013 to 2014 and given to each employee. Participation in the study as well as giving authorization to use earlier HRA results and participation history were voluntary. The data and the material related to the design and implementation of ENSO were retained carefully by the researcher. The study was conducted according to ethical principles of the University of Jyväskylä,[36] and research guidelines provided by the National Advisory Board on Research Ethics in Finland.

RESULTS

Characteristics of the Participants

Most participants were male (83.4%) and local forest officers (47.1%), as summarized in Table 1. Other personnel groups were executives (14.5%), officers (18.9%), terminal workers (12.5%), and lumberjacks (5.8%). At baseline, 586 people completed HRA and 65 declined. At follow-up, proportions were 419 participants and 111 decliners. The selected baseline HRA results were the following: mean body mass index 26.9, systolic blood pressure 137 mm Hg and diastolic blood pressure 79 mm Hg. The mean VO2max results were 33.6 mL/kg/min females and 38.7 mL/kg/min males resulting in an overall fitness class of 4.2 based on the Shvartz and Reibold scale of 1 to 7.[37]

TABLE 1

HRA Participants, Population Characteristics, and Selected Results at Baseline

	Participants	Declined	Total
Baseline HRA N (%)	586 (90.0)	65 (10.0)	651 (100)
HRA 2013–2014	419 (79.1)	111 (20.9)	530 (100)

Design Evaluation

Size

According to the baseline HRA report by the provider,[38]“the program's aim was to avoid low fitness level and overweight, and achieve vital and energetic everyday life.” Aerobic fitness was seen as a key parameter and planned to improve from a moderate level (4.2) to a good level (5.0) in the scale of 1 to 7.[37] The participants were under a different life-change support depending on their subgroup status after the initial HRA: low, moderate, or good health and fitness level (see Table 2).

TABLE 2

Descriptions of the Targeted Life-Change Support for the Three Subgroups

Subgroup	n	% Employee	Age, years	BodyAge, years	Fitness Level (1–7)	Purpose of Targeted Services	Life Change Support	Service Delivery	Number of Individual Contacts	Duration
Low health and fitness	183	31.2%	45.7 ± 10.6	+8.8 ± 2.5	3.1 ± 0.9	To lower health risks	Yes	Individual	12	Over 6 months
Moderate health	168	28.7%	43.6 ± 11.2	+2.9 ± 1.4	4.2. ± 0.9	To improve health	Yes	Group	4	3–6 months
Good health	235	40.1%	42.6 ± 10.9	-3.1 ± 2.8	5.0 ± 1.1	To maintain healthy life-style	No	Group	0	Minimal
Total	586	100%	43.8 ± 11.0	+2.3 ± 5.5	4.2 ± 1.3

The strongest life change support was offered to the population of low health status. The support consisted of 12 individual contacts in over a 6 months’ duration. Employees with moderate health status were encouraged to participate in group sessions with moderate support: five group sessions and four individual contacts with a duration of 3 to 6 months. Employees with a good health status were offered group services with minimal life change support. The more specific descriptions of single size components are presented in appendix 1 (see Supplemental Digital Content 1). As an overview, the duration of the services ranged from a day to several months and the most common method of delivery was a face-to-face meeting or phone calls.

The program aimed “to promote life changes…and with time decrease the number of low health status and increase the number of moderate and good health status.[38]” However, the exact amount of the reduction in low health status was not set in the program plans. Health improvements were expected to occur in BodyAge measurement components,[34] and as a life change process indicator, the TTM was used.[30]

To sum up the size, the provider and the employer shared a common goal of empowering employees in taking care of their own health behavior. The most individual support was offered to the lower health status people.

Scope

According to the design findings, the provider attach great importance to a broad range of services during years 2010 to 2013 (see Appendix 1, see Supplemental Digital Content 1). The services were planned only for the needs of the employees of Stora Enso Metsä; subcontractors or family members were not under a scope. The main focus of the services was to support low-effort, pleasant life changes. Top athlete habits were not emphasized. The nutrition services focused on three main areas: increasing the use of vegetables and fruits, balancing daily meal patterns, and adopting a healthy diet without excess stress. The physical activity part concentrated on easy and safe daily activities, for example, Nordic walking and cycling.

To summarize, the provider established a large scale of services for all employees. The main scope was on life change, nutrition, and physical activity.

Scalability

The provider hoped to capture a 100% reach out to the program across all the 120 units. The aim was to have everyone participate, at least once a year into an annual WHP event or into targeted services, even though the participation was voluntary. Several communication tools were planned to increase awareness of the program and willingness to participate. According to the marketing plans, every local office should be sent a campaign poster during the launching and in 2013. In addition, the plan was to mail materials directly to the employees’ homes: a vitality book in 2011 and service books in 2012 to 2013.

To ensure local awareness to the program, the employer selected 28 voluntary playmakers among the employees to support local participation and a two-way communication between the employees and the provider. The playmaker network was planned to cover all employee so that a single playmaker was responsible for 15 to 30 employees. A playmaker was hoped to be willing to promote wellbeing in their own local area and to have no superior status. The selected playmakers were trained by the provider and every year they had their own task list to be completed.

In order to foster participation, registration of the program wanted to keep as simple as possible. When opening the registration, a marketing flyer (service info, target groups, dates, and places) was released via intranet or email and employees were able to sign into the services via email or the registration website. Finally, the employer was informed of the final amount of participants before implementing the service.

As an overview, multiple communications channels and easy accessibility were deployed to maximize scalability of ENSO.

Sustainability

According to the design data, the continuum of the program was decided annually based on employees’ feedback, employer's needs, and provider's databases. After the decision, the provider planned the program continuum and the services for the next year. From the provider's side, a head coach was assigned to implement services after baseline HRA in 2011. The head coach's job was to ensure implementation according to the annual planning and the budget. The design analysis indicates that the head coach was the only clear resource of sustainability, and budgeted to work 8 hours per week in the ENSO program. However, there was no clear documentation on the actual hours spent. Other work hours by provider, such as meetings, preparation of services, etc., were not included in the sustainability assessment.

From the employer's part, sustainability required resourcing money for on-going services, and the executives’ time and effort to keep the process running. According to the employer's interviews, the plan was to build a new long-term wellness program, but in the design analysis, a clear documentation on how long the program should be was not found. HR executive (resources), an HR manager (wellbeing and safety), and a Communication manager time usage varied between few hours per week to few hours per month during the program.

To summarize sustainability, the provider had their own CRM protocol and service production to keep the process running, but the design material did not explicitly state how much resources both parties were willing to spend in the long-term.

Fit With Best Practice Design Principles

In Table 3, the ENSO design-specific actions are compared with the Best Practice dimensions.[19] Actions were taken on every dimension, meaning that no crucial characteristics were totally missing. However, three divergences should be noted. First, the planning did not include multiyear strategic considerations, which is comparable to sustainability findings. Second, the provider had minimal cooperation with the occupational health care units due to an administrative reason. There were three different occupational health care vendors, and a clear model for knowledge and individual health information transfer was not found between the provider and the three vendors. Third, the ENSO program was designed without engaging the employer's middle management, because the implementation went through the playmakers. Middle managers had the same role as occupational health care: they were informed, but not involved.

TABLE 3

Comparison of ENSO's Design to Nine Best Practice Principles[19]

Dimension	Actions taken	Divergence
LeadershipElements that set the vision for the program, ensure structural and financial support, and engage leaders throughout the organization	- Adequate resourcing annually- Connection to company's mission- Connection to company's Rethink –transformation process	- No clear plan for sustainability- No middle superior support
RelevanceElements that address factors critical to participation and engagement of employees and their families.	- Pleasant life change message- Easy accessibility to services- Tailored services for individuals- Supporting self-care	- No family involvement
Partnership Elements that relate to efforts designed to integrate with multiple stakeholders.	- Use of local resources whenever possible	- No connection to community- No connection to occupational health care
ComprehensivenessElements that meet the definition of a comprehensive program[22].	- Assessment of health risks with feedback- Local playmaker (activator) network- Health education- Connection to workplace safety- Linkage to worksite screenings- Behavior change programs	- Physically and socially supportive environment not available to all units- Playmaker's support not available to all employees (small units)
Implementation Elements that ensure a planned, coordinated, and fully executed implementation.	- Operations work plan- Provider's CRM used as an implementation management system- Targeted outreach	- Long-term planning missing
Engagement Elements that promote respect throughout the organization, build trust, and facilitate program co-ownership.	- Right culture “spirit of wellness”- Leaders care about the health of employees- Low turnover rate, long careers- Long history in annual WHP events- Local playmaker network	- No strong incentives used
CommunicationsElements that include a branding approach, ongoing communications using multiple delivery channels, and targeted messaging.	- Tailored branding- All-year communication actions- Services had their own marketing materials- Intranet, website, postages, flyers
Being data-drivenElements that representthe importance of informed decision making and providing guidance through ongoing measurement, evaluation, reporting, and analytics.	- On-going measurement and evaluation- Analysis and reports- Data security- Data used for designing services	- No data transfer to occupational health care
ComplianceElements that ensure that the program meets regulatory requirements and safeguards data in the individual level.	- Data confidentiality- Compliant with laws and regulations

The strongest alignment to the guidelines[19] was found in relevance, comprehensiveness, implementation, engagement, communications, emphasis on data usage, and compliance to regulatory issues. According to the collected design data, the provider and the employer both emphasized the use of HRA, tailored services for individuals, a solid life change message, multiple communication tools, and data-driven execution as primary necessities for ENSO.

Penetration

The penetration rates of this study are represented in Table 4. The employer and the provider estimated the amount of employees who received invitations to participate in ENSO's services. The invitations were sent by mail, called by phone, emailed, made as calendar reservations, informed via intranet, or sent via playmakers. The annual penetration rate was 100% in 2009 to 2010, little less in 2011, and almost 100% again in 2012 and in 2013. In total, this results in a coefficient of 99.5%.

TABLE 4

The Penetration Coefficient Calculation of ENSO

		Year
		2009	2010	2011	2012	2013
The amount of employees reached with invitations	Employer's evaluation	15	651	626	630	625
	Provider's evaluation	15	651	619	620	623
Penetration calculation	The employer's and the provider's estimations in average	15	651	622.5	625	624
	Target population	15	651	634	630	625
							Total
	Penetration coefficient (%)	100	100	98.2	99.7	99.8	99.5

Several services reached a coverage of 100% (see appendix 2, Supplemental Digital Content 2). The annual WHP events, fitness camps, HRAs, and playmakers’ training camps reached the highest level every year. The Vitality Book of 2011 and the Service Books of 2012 and 2013 were sent directly to the employees’ homes and none were returned to the sender. The only services with lower penetration rates were wellness coaching in 2011 to 2012, targeted phone calls in 2012, and lumberjack wellness coaching in 2013. These three services had a combined E-mail and phone invitation protocol, and during early years of ENSO, not all the employees were voluntary to give out their phone numbers, which increased the number of missed phone calls.

The primary channels for invitations were targeted emails, direct contacts made by the playmakers, and the employer's intranet. The use of email as an invitation tool was simple and accurate among executives and officers. The emails were sent partly by the employer and partly by the provider. However, there was not possibility to follow-up how many of these emails were opened and read by the recipients, and readability check-up tools were not used in this study.

Implementation

Implementation demonstrates how well the program was delivered to the target population as planned. The implementation coefficient was calculated by dividing actions executed by all intended actions (see Fig. 2).

FIGURE 2

Planned versus executed actions of ENSO during years 2009 to 2014.

Altogether, 73 actions had been planned for the years 2009 to 2013, of which 59 were executed, resulting in a total coefficient of 80.8% for implementation. Of planned actions, 44 were services, four were HRAs, 12 were communication tools, nine were materials, and four were cooperation ideas. The highest implementation percentage was found in HRA's and communication tools, where all planned actions were executed. In services, 33 of 44 planned services were executed resulting in a 75.0% coefficient. The main reason for nonexecution was insufficient number of participants. In some cases, services were planned, but never received an implementation authorization by the employer. In materials, eight out of nine were implemented as planned, for example, the Vitality Book, service books, and HRA reports. In 2013, there was a plan to hand out every employee the My Path coaching book, but the employer did not see it reasonable due to the statutory negotiations that were just about to start. Two out of four cooperation actions were implemented, including the employer's development meeting of the subcontractors and the manager info in 2011. Meanwhile, the executive infos in 2011 and a closer cooperation with occupational health care units in 2012 were never completed.

DISCUSSION

This article investigated a design and implementation process of a real-world WHP program (referred to as ENSO) by using 4-S and PIPE Impact Metric models.[23] The findings were compared against nine Best Practices dimensions.[19] To best of our knowledge, this was the very first evaluation of a design process in a multiyear market-based program. The design evaluation tools (4-S), size, scope, scalability, sustainability, and the execution evaluation components penetration and implementation from the PIPE Impact Metric model offered a feasible framework for a mixed-methods data analysis.

The program's size was clearly set for three subgroups according to the participants’ physical condition (low, moderate, good health), and the scope included a large set of services to contribute to pleasant and low effort lifestyle changes that would still be sufficient enough to improve the employees’ health and wellness. The evidence regarding a wide range of actions is ambiguous. Multiple actions have been supported by Shain and Kramer,[17] because a program should meet the requirements of a wide variety of participants. Lerner et al[4] found a wide-range content problematic in an economic impact perspective, which was not evaluated in this study. Furthermore, the body of literature suggests that targeting actions only to poor health status would increase ROI.[2-5,12] Recently, Batorsky et al[39] found a connection between a moderate level of services and effectiveness. However, the provider in ENSO favored a broad scope and chose to target all employees. The idea of addressing multiple “entry channels” simultaneously had been supported earlier, because it facilitates individuals’ involvement in the program[18] and emphasizes relevance to participants.[19] Clearly, the decision to target all employees with several services is not supported with one consent in the literature. The analysis of the last components of the PIPE Impact metric, the participation and the effectiveness, is needed to brighten a final judgment of design decisions behind ENSO.

The ENSO program started with a HRA and feedback. HRA is a common and effective tool in WHP programs,[6,9,18] but not sufficient by itself.[16] Soler et al[16] found strong evidence of effectiveness, when HRA was conducted in combination with health education or connected to targeted services. These findings support the actions selected in ENSO.

In the literature, scalability and penetration are connected in the sense that high penetration measures the success of scalability design.[23] ENSO aimed at 100% scalability in a decentralized organization using several communication tools. The penetration coefficient was at a high level of 99%. This was mostly due to the explicitly known target population. Aziz et al[25] reported penetration rates between 1% and 100% in diabetes prevention programs, but only a minority of studies report an exact target population. O’Reilly et al[26] estimated their penetration rate as low as 4%. In well-known target populations, such as in ENSO, planning process should seek ways to achieve 100% penetration rate.

In ENSO, 81% of the planned actions were implemented. This result can be seen as a preliminary for a market-based WHP program. Study findings indicate that in a multiyear real-world setting, implementation rarely follows program planning completely, which corroborates previous researches supporting the need for process evaluation.[9,23] Interestingly, the recommendations in the literature have emphasized the need to ensure fidelity of program implementation,[6,9,18,19] but implementation calculations are not routinely reported in published WHP interventions. As a benchmark result in primary care, implementation levels over 60% had led to positive outcomes and few studies obtain fidelity levels over 80%.[40,41] Future studies should report implementation rates more clearly and investigate the connection between implementation rates and effectiveness calculations.

On the basis of the comparison between the design process and the nine Best Practice principles, the strengths of ENSO were comprehensiveness, relevance to employees, solid life-change message, multiple communication tools, and data-driven execution.[19] Engagement and “culture of health” was at good level already before ENSO started, but it strengthened even more because of the tailored program branding, shared design process with HR-executives, and the establishment of the local playmaker network. The culture had been correctly identified a priori when launching the program[20] and it encouraged participation and health improvement in a long-term context.[2,9] However, WHP programs are known to fail if they lack leadership support.[9,18,19] ENSO succeeded partly in this principle. HR executives were strongly involved in the program, but the middle management was not. This might have influenced the program's overall engagement and effectiveness.

A major divergence to the Best Practice guidelines was a minimal cooperation with occupational health care. According to the provider, a closer relationship with the occupational health care units was pursued in 2012, but it did not lead to any actions due to administrative reasons. A successful cooperation might have been a valuable resource in promoting health in the low health status group.

Other major divergence was sustainability. Effective programs require ongoing monitoring and maintaining,[6,9] but robust long-term actions were not emphasized in the design process of ENSO. From the WHP provider's perspective, resourcing sustainability might be problematic, if the provider lacks of experience on what to expect from a multiyear process. On the contrary, in market-based context, it is understandable that employers define resources only after a pilot period and after positive user experience. However, the annual decision-making on the program continuum did not help to allocate resources for the ENSO's future. Study findings indicate that sustainability should be defined as a crucial objective already in the design process, because benefits rarely occur before 3 years.[5,6,9,18]

There are two factors that limit the generalization of the results of this study. First, this was the first time the 4-S and PIPE Impact Metric frameworks were used in a WHP context. In addition, the recommendations for “well-designed” WHP programs are young, and more research is needed in order to understand connections between the quality of planning and the expected outcomes. Second, a methodological limitation was that the researcher was closely designing and implementing the ENSO program. This helped to gather a large amount of information from the provider and the employer, but a similar confidential knowledge-transfer might not be possible for an external research group. The strengths of this study were the investigation of a real-world multiyear design and implementation processes, the program and its services were described in detail, and the gathered information treated ENSO as “a whole” combining research and practice.

To summarize the study results, most of the program content was executed as planned (81%) and there were no major deficiencies in the design of ENSO compared with present literature recommendations. However, analyses on participation and effectiveness will be needed in order to measure final outcomes of the program. Overall, study findings indicate that WHP market would benefit the use of evidence based design principles for benchmark and quality improvement purposes and WHP providers should intentionally prepare for long-term actions already during the planning.