Measuring success in obesity prevention: a synthesis of Health Promotion Switzerland's long-term monitoring and evaluation strategy.

AIMS: Since 2007, Health Promotion Switzerland has implemented a national priority program for a healthy body weight. This article provides insight into the methodological challenges and results of the program evaluation.
METHODS: Evaluation of the long-term program required targeted monitoring and evaluation projects addressing different outcome levels. The evaluation was carried out according to the Swiss Model for Outcome Classification (SMOC), a model designed to classify the effects of health promotion and prevention efforts.
RESULTS: The results presented in this article emphasize both content and methods. The national program successfully achieved outcomes on many different levels within complex societal structures. The evaluation system built around the SMOC enabled assessment of program progress and the development of key indicators. However, it is not possible to determine definitively to what extent the national program helped stabilize the prevalence of obesity in Switzerland.
CONCLUSION: The model has shown its utility in providing a basis for evaluation and monitoring of the national program. Continuous analysis of data from evaluation and monitoring has made it possible to check the plausibility of suspected causal relationships as well as to establish an overall perspective and assessment of effectiveness supported by a growing body of evidence.

Initial Situation

Although Switzerland has a very low overweight and obesity prevalence rate in comparison with other European countries [1], a rate that has also stabilized in recent years, a large portion of the country's roughly 8 million citizens struggles with weight problems: approximately 32% of women and 51% of men in Switzerland are overweight [2]. Among school-age children, taking all grades together, around 17% are overweight – though older children, children whose parents are less educated, and children of migrants are more likely to be overweight [3].

In 2007, Health Promotion Switzerland launched a national prevention program for a healthy body weight, aiming to increase the percentage of the population with a healthy body weight. However, Switzerland's federalist state structure presents challenges to such a strategy, as it must contend with 26 separate cantonal health and education systems as well as numerous different state and private stakeholders at the national, cantonal, and municipal level – whereby the country's cantons and municipalities possess a relatively high degree of autonomy in the system. From the very beginning, the national program is confronted with the difficulty of needing, on the one hand, to account for diverse context factors, structures and stakeholders’ requirements, and, on the other, to establish mechanisms of supervision, quality assurance and impact monitoring at the national level. The program faced the challenge of trying to introduce a new (in those days) priority area within this complex overall framework, gaining the cooperation of diverse stakeholders, and initiating and implementing sustainable, health-promoting processes within societal structures.

The national program was launched with an umbrella campaign to raise awareness and to mobilize the population and political stakeholders. Parallel to this, cooperation with the cantons was established. The resulting Cantonal Intervention Programs came to form the core of the national prevention program. Other program cornerstones included processing evidence [4,5] and coordinating stakeholders. As analysis of the evidence revealed that early intervention is especially promising [4], the national program focused on the roughly 1.4 million children aged 0-16 years in Switzerland as well as their caregivers.

Cantonal Intervention Programs for a Healthy Body Weight

The Cantonal Intervention Programs are designed to counteract the factors contributing to overweight among children and youths. By promoting a balanced diet and adequate exercise, the aim is to stabilize the share of children with a healthy body weight in the medium term and to increase it in the long term. In addition to the children and youths themselves, the program's target groups include caregivers such as parents, teachers, daycare supervisors, etc. Each intervention program is based on four pillars: intervention projects, policy measures, public relations, and networking/coordination.

Acknowledging the long-term nature of possible impacts, the national program has been arranged to last 12 years, subdivided into 3 multiyear phases.

In total, more than 200 projects are being implemented based on over 100 project concepts [6]. They are being carried out in highly diverse settings, encompass a variety of intervention approaches, and address many different target groups overall. For example, school environments are being adapted to encourage more physical activity, mother/father counselors are being trained, parents are being given guidance, and dietary recommendations are being developed.

This comprehensive intervention strategy is consistent with scientific evidence showing that successful prevention of obesity requires multilevel approaches lasting several years, integration of a wide range of state and non-state actors, adaptation of intervention programs to local contexts, and state supervision [5,7]. These findings are supported by a recent systematic review and meta-analysis showing, among other things, that school-based intervention programs are particularly successful when combined with community-level approaches [8]. The European Charter on Counteracting Obesity accordingly promotes a comprehensive societal approach that calls for broad engagement, greater political guidance, and legally anchored regulatory measures [9]. Today, most European countries have national action plans for obesity prevention that bring together various intervention approaches under one roof [5].

Since several million CHF are invested in the priority program in Switzerland every year (in 2011, Health Promotion Switzerland invested CHF 4.6 million while the participating cantons contributed a combined CHF 10.2 million), the national program has been accompanied by efforts towards evaluation since the very beginning.

Evidence Base and Methodology

Challenging Complexity

Evaluation of the program faces many obstacles due to the complexity of the Cantonal Intervention Programs, which are located in differing political, economic, and social contexts, bringing together a variety of stakeholders and interests and combining an assortment of intervention approaches in highly diverse settings. To help structure the various questions arising from this complexity, program organizers turned to the Swiss Model for Outcome Classification (SMOC) [10] (fig. 1). Based on the outcome model developed by Don Nutbeam [11], the Swiss model provides a basis for classifying the impacts of health promotion and prevention. It is one of several health-specific classification models that have been formulated – including the precede-proceed model from Green and Kreuter [12] and the health promotion framework from Saan and de Haes [13] – and therefore differs from the type of thematically unspecific basis models that are often used to plan and evaluate intervention programs (e.g. [14,15]). Thanks to its detailed, health-specific formulation of categories, the SMOC offers the advantage of a theory-based reduction of complexity [16]. It enables classification of measures and planned or achieved impacts of health promotion or prevention projects, making it possible to assign them to theory-based categories, and allows possible causal relationships to be explored. The model comprises four columns of information capturing measures (A) and outcomes (B-D). The ultimate target impacts of the health promotion and prevention program – expressed as health indicators – are assigned to column D. The health determinants are found in column C, while the factors influencing these health determinants are located in column B.

Fig. 1

The categories and subcategories of the SMOC [10].

Due to the significant heterogeneity and the limited predictability of developments, the present priority program was not designed chiefly according to a fixed, desired outcome, but rather according to key sets of questions. Figure 2 shows the main elements of the national program in the leftmost column: the Cantonal Intervention Programs and their four pillars – policy measures, networking, communication, and intervention projects – as well as the public awareness campaigns. The key questions pursued are displayed in the three columns to the right. These encompass components of both process and impact evaluation, ranging from performance-, acceptance-, and use-related questions underlying the measures to key epidemiological questions. The arrows in figure 2 illustrate – similar to an outcome model – suspected causal relationships occurring at a higher level.

Fig. 2

Key questions within the different levels of impact of the national prevention program.

It is not possible to use experimental or quasi-experimental studies to conduct a comprehensive impact evaluation of the cantonal-level programs. This is because – in addition to fundamental reservations in view of the prevailing complexity (see, e.g. [17,18]) – the programs differ too much and there are too few cantons that may be used as experimental controls. By contrast, individual subprojects, in principle, may be investigated using a control group study design, and this has in fact been done to a certain extent [19,20,21]. However, ad hoc evaluations of this sort only account for a small fraction of the total activities implemented and are typically unable to verify effects on the level of individual body weight. Effects on such long-term outcome levels are only likely to be seen after a number of years, brought about by interactions between the many different interventions implemented [22,23].

Evaluation System

In order to assess this priority program, a combination of different monitoring and evaluation projects was set in motion (table 1). These generate data at regular intervals that may be used to assess developments on different outcome levels. Whereas the monitoring projects continuously track the development of relevant indicators, the evaluation projects seek to go even deeper and investigate possible causal relationships. The indicators used in monitoring and evaluation concern a wide array of outcome levels and timeframes. The evaluation system observes indicators at the level of individual behavior as well as indicators on structural and political levels, taking various perspectives into account (population, media, political stakeholders).

Table 1

Key elements of the evaluation system regarding exercise, nutrition, and body weight

	Research projects	Evidence base and activities to date	Latest report a
Monitoring	population survey	representative population surveys on knowledge and attitudes with regard to nutrition, physical activity and body weight conducted by telephone with about 1,000 people (2006, 2009, 2012)	[24]
	monitoring policy developments	monitoring of political developments in national, cantonal, and selected city parliaments (annually since 2008)	[25]
	media monitoring	analysis of reporting in selected Swiss print media publications (three times every six months, between 2006 and 2008)	[26]
	monitoring of conditions in school settings BMI monitoring	online survey of school principals in all public schools in 22 of 26 Swiss cantons (2009, 2013)
measurement of the body mass index among children and youths in three age groups (annually since 2006, in three cities; in 2010 and 2013, including additional data from five and eight cantons, respectively)	[27]
[3, 28]
	monitoring of individual indicators	review of selected indicators on the basis of secondary analysis of existing data sources (since 2007, annual updates)	[29]
Evaluation	evaluation of the cantonal intervention programs	synthesis of the self-evaluations of the Cantonal Intervention Programs and of Health Promotion Switzerland. Interviews with those responsible and review of program reports (2008, 2010, 2012)	[6]
	evaluation of pilot projects	comprehensive process and impact evaluations of five pilot projects to date (e.g. in 2011: Evaluation of the project Youp'là bouge)	[19]
	evaluation of public campaigns	representative population surveys on the awareness and mobilization campaigns implemented in the initial years (2007, 2008, and 2009)	[30]
	stakeholder surveys	telephone survey of key stakeholders and, since 2009, online survey of cooperating partners (2005, 2006, 2009, 2013)	[31]
Other studies	synthesis of impact evaluation of module projects	synthesis of the impact evaluations of 12 selected module projects of the Cantonal Intervention Programs (2014)	[32]
	study on the prevalence and costs of weight-related illnesses	analysis of data from health insurers CSS (2008)	[33]

The literature references refer to the most recently published relevant report. The reports (with exception of the internal report [31]) are available in German and/or French at .

The 12 monitoring and evaluation projects employ different research methods that cannot be fully described here (see table 1). Methodological details may be found in the respective reports available for download at . In the case of monitoring conditions in school settings, for example, an online survey of principals in 22 Swiss cantons was conducted, requesting information about the structural anchoring of measures to promote a balanced diet and adequate exercise. The 916 principals surveyed (38% response rate) represented 8,738 public institutions of compulsory education and approximately 760,000 students [24]. In the case of BMI monitoring, as another example, body weight and height measurements collected (primarily) by school doctors were statistically evaluated for three age groups of schoolchildren. In 2013, these evaluations incorporated the BMI data of 26,707 schoolchildren from 11 cantons and cities [25].

The monitoring activities of Health Promotion Switzerland were one of the cornerstones of the comprehensive Swiss Monitoring System on Nutrition and Physical Activity (MOSEB), which was established in 2008 in conjunction with the National Programme of Nutrition and Physical Activity (NPNPA).

Arranged according to the outcome model's various impact levels, the different monitoring and evaluation projects continuously generate answers to overarching key questions and help establish an overall picture of societal developments and the progress of the program (fig. 3). Use of the outcome model helps to define evaluation priorities, identify gaps, and use existing resources efficiently [34]. As recommended by Patton [18] and others in regard to complex programs, the evaluation system is continually adapted in order to address new key questions as they arise.

Fig. 3

Monitoring and evaluation projects of Health Promotion Switzerland in the areas of exercise, nutrition and body weight.

Selected Content-Related Findings

In the following, selected results from the different monitoring and evaluation projects (table 1) are briefly described to provide insights into the current situation and ongoing developments.

Awareness Raising and Political Mobilization

Almost all of Switzerland's cantons were willing to commit to a long-term, coordinated engagement on behalf of healthy body weight. Out of 26 cantons, 22 implemented a multiyear Cantonal Intervention Program between 2007 and 2011, while 20 have committed themselves to a second 4-year program phase (2012-2015) – though the latter hinges on the continued political support of the respective government. However, beyond this, topics of nutrition, exercise, and healthy body weight have made few appearances on the agendas of the cantonal parliaments or the Swiss Federal Parliament. Legislative initiatives that would restrict individual freedom are highly unlikely to appeal to a majority [25] and also fail to garner support among the population [24]. Regarding government measures, population surveys show a mixed picture overall: on the one hand, a majority of those surveyed (71% in 2012) consider ensuring balanced nutrition, adequate exercise, and a healthy body weight to be a matter of personal – not state – responsibility; on the other hand, a majority also express support for assistance measures on behalf of balanced nutrition and exercise as long as these do not involve bans or taxation [24].

In regular surveys, program stakeholders have underscored the importance of issues of obesity and health [31]. Within the broader population, people's perception of weight-related issues as a societal problem has been successfully increased: whereas 40% of respondents in a 2006 population survey felt that something should be done to address obesity, that number rose to 51% in 2012 even though the prevalence of obesity in the population remained stable over the same period [29].

Knowledge in the Population

As revealed in the representative surveys, a majority of the population understands the importance of exercise and nutrition for a healthy body weight, and a majority also understands the main health problems, in particular cardiovascular diseases (75%, 2012) and diabetes (59%, 2012), that may result from being overweight. However, crucial differences exist between population groups arranged according to education level. A general increase in knowledge among the broader population was not identified over the period in question. With regard to personal exercise and dietary behavior, a considerable discrepancy was observed between people's perception of themselves and others. A majority of respondents felt that they exercised more and ate healthier than the rest of the Swiss population [24]. Monitoring of the media additionally showed that a majority of the coverage was consistent with current prevention recommendations [26].

The Potential of the Cantonal Intervention Programs

In the 22 cantons that conducted intervention programs for a healthy body weight, a wide array of projects was implemented in diverse settings, targeting diverse age groups. The projects – which Health Promotion Switzerland had recommended to be multiplied based on quality assessments – were widely taken up; only one third of the projects were newly developed by the cantons in the context of the intervention programs. Networking between the cantons and linguistic regions increased markedly, and their communication messages were successfully harmonized. In just under half of the participating cantons, the respective Department of Education supported its Cantonal Intervention Program, enabling better institutional anchoring. Structural changes have occurred in a majority of the projects implemented. The percentage of condition-oriented projects (as opposed to behavior-oriented) increased from 56 to 69% between 2009 and 2011. Across all levels, the perceived importance of equal opportunity has increased: the share of projects addressing the issue of equal opportunity (including aspects of social inequality, migration, and gender) grew markedly between 2009 and 2011, from 20 to 35% [6]. An impact analysis of the 12 most far-reaching projects showed that they reached roughly 174,000 children and 9,500 multipliers, encompassing approximately 6,400 settings. Program evaluators extrapolated that these 12 projects brought about cultural changes – such as a new, jointly supported ‘food culture’ within a given school – in around 3,200 settings and triggered structural changes – such as improved spaces for physical activity in kindergartens – in approximately 1,600 settings. An increase in relevant knowledge occurred among an estimated 32,000 parents, children, and youths, while a positive change in attitude occurred among an estimated 52,000 individuals. It was further estimated that, thanks to these 12 selected projects, around 74,000 children and youths began eating more healthily and approximately 47,000 children and youths started getting more exercise [32].

Changes in School Settings

The surveys of principals showed that the majority of schools in Switzerland were engaged in the areas of exercise and nutrition: half of the schools furnished teachers with practical recommendations related to these areas; and more than half of the schools provided parents and children with recommendations for healthy snacks, offered voluntary school sports – in addition to obligatory recreational activities -, and discussed the issues at teacher conferences. However, 70% of the schools did not consider health promotion or prevention among their highest, overarching priorities, and did not regard nutrition or exercise as pressing issues requiring individual attention. Facilitating healthy schools according to a holistic approach, violence prevention, and early detection/early intervention were considered higher priorities by most schools. Correspondingly, integrating the issues into a broader health-promoting school strategy was found to be very important, as is being done by the Swiss Network of Health Promoting Schools. Despite the multipronged efforts of the Cantonal Intervention Programs in schools, the national survey failed to find evidence of any increase in overall structural anchoring of the issues in schools between 2009 and 2013 [27].

Exercise and Dietary Behavior of Children and Youths

According to the study ‘Health Behavior in School-Aged Children’, conducted by the foundation ‘Sucht Schweiz’, only 12% of schoolchildren aged 11-15 years are physically active for at least 1 h every day, as is currently recommended by the Swiss Federal Office of Public Health [35]. Nevertheless, a reported 45% of boys and 33% of girls are physically active for at least 1 h per day on 5 days of the week, at minimum. According to the HBSC study, only two fifths of children between 11 and 15 years consume fruits and vegetables every day. A majority of the children surveyed were found to adhere to the recommendations for consumption of water and unsweetened drinks as well as fast food. However, only 21% fulfilled the recommended vegetable intake, 15% adhered to the recommended fruit intake, and 35% were found to consume no more than the recommended daily maximum of sweets (including colas and other sweetened beverages) and salty foods [35]. These investigated indicators of nutrition and exercise behavior have changed little over the last two decades. One exception is consumption of fruits and vegetables, which increased slightly between 2002 and 2010 [35].

Obesity among Children and Youths

According to the results of BMI monitoring, when taking all of the included regions and grade levels together, approximately 17% of children and youths in Switzerland may be considered overweight (including obese), and almost 4% obese. Children and youths whose parents have a lower education level and children and youths of foreign nationality are disproportionally affected by problems of being overweight or obese. As the measured BMI data of children and adolescents in the cantons and cities monitored show, the prevalence of overweight children and youths has stabilized at this relatively high level in recent years [28], following a large increase in the proportion of overweight children in the 1980s and 1990s [3].

Methodology-Related Findings

To a certain extent, the SMOC helps to account for the inherent complexity of the issues, making it easier to grasp key dynamics on behalf of program steering and evaluation. The data generated by monitoring and evaluation efforts provide a good overview of selected aspects of exercise, nutrition, and body weight in Switzerland – they are highly relevant for steering and further developing the national program. The periodically generated findings supply the national program and enable continuous learning. The data from monitoring and evaluation illustrate the overall progress and challenges in the four pillars of intervention projects, policy, networking and public relations, and indicate possible causal relationships. They show, for example, how the coordination of key stakeholders has been improved across cantonal and linguistic borders over the years, and how this has increasingly made it possible to scale up high-quality projects. Impressive changes have been identified on intermediate outcome levels – changes that very likely result from the measures implemented. For example, many school cafeterias have adapted their food offerings to obtain a label of balanced nutrition, many kindergartens and schools have arranged their break rooms and classrooms in a way that promotes physical activity, and several thousand multipliers have been able to enhance specific skills in specialized courses.

The available data do not enable observers to determine whether the national program is responsible for the recent stabilization in obesity rates (attribution problem) or how exactly the program has contributed to the observed changes (contribution problem). The development of people's body weight is influenced by such a wide array of biological, social, material, and economic factors (cf. [36]) that it is impossible to prove causal relationships at this national level. However, based on theoretical reflection and the sheer scope of the multiyear program, a corresponding causal link can be assumed. There have certainly been developments that appear directly connected to program measures, but many suspected links weaken as the spatial and temporal distance is increased between possible cause and effect. The fact that the majority of the projects implemented are now conceptually focused on structural changes in settings and that they increasingly reflect and factor in the importance of equal opportunity would appear to indicate, based on the available literature [4,5], that the programs are on the right track to make a decisive, sustainable contribution. Indeed, the observed structural and cultural changes support this conclusion. Equally impressive is the estimated tens of thousands of children and youths that were reached in a single year and persuaded to change their behavior thanks to the program's 12 most far-reaching projects. Nevertheless, additional analyses are needed in order to better assess the extent of the national program's contribution to the observed societal changes. To this end, Contribution Analysis [37], an iterative method for testing the plausibility of causal relations established by John Mayne, appears to be a particularly promising approach.

When developing the evaluation system, it became apparent that the access to and the availability of certain desired data is restricted. At first glance, there appears to be a lot of data available regarding the issues [38]. Upon closer examination, however, numerous gaps and quality problems are revealed. For example, there are scarcely any measured empirical data available on the exercise and nutritional behavior of children and youths, and the age range covered by national surveys is very limited (the HBSC study, currently the most comprehensive data source on the nutrition and exercise behavior of children and youths, only covers ages 11-15 years). Analyses comparing survey results with corresponding empirical measurements have revealed such high levels of deviation [39,40] that the results of most available studies must be interpreted very carefully. In addition, individual sources of data operationalize the same issues in different ways, further complicating efforts at comparison. Data on structural indicators remains scarce and very difficult to gather.

There is still enormous potential in linking different sources of data. It would be interesting, for example, to compare the data from surveys of principals with the data from BMI monitoring, in order to determine whether the long-term prevalence of overweight students is lower in those schools that undertake significant prevention efforts. Up to now, it has scarcely been possible to establish such links due to limited availability of data, issues of data protection, and methodological or practical limitations.

The costs associated with the surveys are high. Every year, Health Promotion Switzerland invests approximately CHF 400,000.- in various monitoring and evaluation projects. Whereas monitoring projects are mainly associated with large research expenditures, evaluation projects are additionally associated with costs arising from the activities of stakeholders in the field: they participate in project design, support implementation, often contribute to the pool of data with their own surveys, and aid in the interpretation of findings. As a result, the evaluation projects frequently encounter human resource constraints that can only be overcome with persuasive efforts and the prospect of mutually beneficial results, reinforcing stakeholders’ willingness to continue supporting evaluation activities. The evaluation system developed over the years is largely the result of complex processes of negotiation with stakeholders from scientific and practice-oriented circles.

Conclusions

The success of a complex national strategy cannot be evaluated solely on the basis of a handful of indicators, such as the BMI. A wide variety of assessment measures were used to evaluate the Swiss national program for a healthy body weight. These revealed the following:

The share of overweight children and youths in Switzerland has stabilized since 2006. Whether, and to what extent, the national prevention program has contributed to this development cannot be determined conclusively, in particular due to the complexity of causal relationships involved. However, based on the observed effects at various intermediate levels, it appears plausible that the national program has made a significant contribution.

Despite the rapid, encouraging progress seen at intermediate levels, it takes many years for successful approaches to demonstrate their full potential for impact in a complex field like obesity prevention. So far, the experience of the program has reinforced the idea that long-term success requires context-specific multilevel approaches whose duration of effectiveness extends over several years.

Health Promotion Switzerland's outcome model has shown its ability to illustrate complex links while simultaneously drawing attention to core issues. It helps distinguish the different levels of impact, maintain an overview of the whole system, and facilitates better communication about impacts – also within the political process. The mass of data emerging from monitoring and evaluation enables observers to better understand developments in the monitored subsystems and makes it possible to examine the plausibility of causal relationships.

The various data collected provide interesting clues as to the development of the priority program, but they do not speak for themselves. The data must be jointly considered and interpreted by all stakeholders in order to reach appropriate conclusions and make decisions. Further, the evaluation system itself must be continuously adapted in order to reflect new developments.

Disclosure Statement

One of the authors (Ackermann, G.) works part time as an independent consultant. He uses the outcome model in his consultancy and received some funding from Health Promotion Switzerland for this article.

The other authors do not have any significant financial interest/arrangement or affiliation with any organization/institution whose products or services are being discussed in this paper. The authors do not receive any sponsorship or funding to their research.