Sustainable Undernutrition Reduction in Ethiopia (SURE) evaluation study: a protocol to evaluate impact, process and context of a large-scale integrated health and agriculture programme to improve complementary feeding in Ethiopia.

INTRODUCTION: Improving complementary feeding in Ethiopia requires special focus on dietary diversity. The Sustainable Undernutrition Reduction in Ethiopia (SURE) programme is a government-led multisectoral intervention that aims to integrate the work of the health and agriculture sectors to deliver a complex multicomponent intervention to improve child feeding and reduce stunting. The Federal Ministries of Health and Agriculture and Natural Resources implement the intervention. The evaluation aims to assess a range of processes, outcomes and impacts. METHODS AND ANALYSIS: The SURE evaluation study is a theory-based, mixed methods study comprising impact and process evaluations. We hypothesise that the package of SURE interventions, including integrated health and agriculture behaviour change communication for nutrition, systems strengthening and multisectoral coordination, will result in detectable differences in minimum acceptable diet in children 6-23 months and stunting in children 24-47 months between intervention and comparison groups. Repeated cross-sectional household surveys will be conducted at baseline and endline to assess impact. The process will be assessed using observations, key informant interviews and focus group discussions to investigate the fidelity and dose of programme implementation, behavioural pathways of impact and contextual factors interacting with the intervention. Pathways of impact will also be explored through statistical analyses. ETHICS AND DISSEMINATION: The study has received ethics approval from the scientific and ethical review committees at the Ethiopian Public Health Institute and the London School of Hygiene and Tropical Medicine. The findings will be disseminated collaboratively with stakeholders at specified time points and through peer-reviewed publications and presentations. © Author(s) (or their employer(s)) 2018. Re-use permitted under CC BY-NC. No commercial re-use. See rights and permissions. Published by BMJ.

The mixed methods evaluation comprises impact and process evaluation to support course correction and interpretation of primary outcome results.

The study will also provide evidence on drivers of diet, agriculture and child feeding-related behaviours in the context of a large-scale behaviour change intervention.

A quasi-experimental study design is used and Ethiopia has a very active nutrition landscape in which the roll out of similar interventions into comparison areas may occur over the study period.

Introduction

Linear growth failure or stunting in children under 5 years of age reflects suboptimal nutrition or health conditions in a population and negatively impacts cognitive development.1 Growth failure is most pronounced between 6 and 23 months of age, during which period poor infant and young child feeding (IYCF) practices and disease drive stunting in low-income settings.2 Though optimal child feeding practices are essential to child survival, growth and development, globally 36% of children 6–8 months have not yet begun to eat complementary foods and 85% of children aged 6–23 months are not fed the minimum acceptable diet (MAD) (comprising minimum dietary diversity and number of meals per day).3

WHO recommends that behaviour change communication is made central to strategies to improve IYCF.4 These interventions seek to influence the knowledge, attitudes, beliefs and social norms that underpin dietary practices.5 Child feeding education targeted to caregivers of children under 24 months increases height gain and, in food insecure populations, decreases stunting6 7. Consequently, the promotion of breastfeeding and complementary feeding have been identified by the Lancet Maternal and Child Nutrition Series (2013) as key actions to reduce undernutrition.7 Behaviour change interventions may also provide opportunities to link nutrition education to other actions within a broader framework to address stunting.8

The intersection between low dietary diversity and poor complementary feeding practices forms an important nexus for multisectoral programming in subsistence agriculture settings. Diet quality is an essential component of complementary feeding9 and agriculture has an important role to play to ensure the availability, access and use of diverse foods.10 The promotion of nutrition-sensitive agriculture practices can also enhance food security and women’s empowerment to accelerate progress in IYCF.11

Country context

An estimated 37% of Ethiopian children under 5 years have stunted growth.12 IYCF practices are poor and dietary diversity is among the lowest in the world.13 Nationally, 7% of children 6–23 months old consume the MAD and initiation of complementary feeding is delayed in 51% of children.12

Ethiopia builds on more than a decade of experience adapting tools and developing message delivery platforms to promote breastfeeding and, increasingly, complementary feeding.14 15 Prior research has identified that barriers to improved complementary feeding include poor diversity of consumption at household level, social norms related to animal source foods, insufficient paternal engagement and low women’s empowerment.16 17 The government has recognised that leveraging contributions from both health and agriculture sectors is necessary to accelerate progress in child feeding and stunting reduction.

Ethiopia benefits from professionalised health and agriculture sector platforms that deliver government programming to every district and kebele (administrative subdistrict) in the country. Health extension workers (HEWs), women deployed at a ratio of approximately 1:250 households, oversee routine health post and community-based health and nutrition services. Agriculture extension workers (AEWs), largely men and provided for at a ratio near 1:167 households, deliver farmer training and demonstration gardens.

Nutrition policy takes a broad multisectoral framework.18 The National Nutrition Programme maps key actions identified across multiple sectors to improve the nutritional status of target groups and to reduce stunting.19 Federal guidelines have also been formulated to support nutrition governance through the formation of multisectoral coordination committees at each level of government.20

The intervention

The Sustainable Undernutrition Reduction in Ethiopia (SURE) is a government-led multisectoral programme to integrate health and agriculture sector service delivery for nutrition outcomes. It is a complex intervention that both incorporates and enhances the former community-based nutrition (CBN) programme by expanding nutrition services and integrating selected agricultural services. Service delivery is underpinned by actions to strengthen systems and to facilitate multisectoral coordination.

CBN has been an important national health initiative since 200821. The main components include: (i) monthly growth monitoring and promotion for children under 24 months; (ii) vitamin A supplementation and deworming and (iii) quarterly screening for acute malnutrition. SURE aims to enhance CBN by expanding services to improve complementary feeding and dietary diversity. Using the socioecological theory of behaviour change, the SURE package includes components targeted to individual, social, organisational, community/societal and policy spheres22 (see table 1).

Table 1

SURE programme components and subcomponents

Component	Subcomponent	Frequency	Primary target	Delivered by
Community-Based Nutrition Services	Growth monitoring and promotion	Monthly	Mother	HEW*
	Screening for acute malnutrition	Quarterly	Child/mother	HEW
	Vitamin A supplementation and deworming	Biannually	Child	HEW
Enhanced Nutrition Services	IYCF† counselling and agricultural advising	Once/2 months	Father and mother	HEW and AEW‡
	Men’s and women’s groups	Once/month	Community	HEW or AEW
	Media messages	Routine	Community	Broadcast media
	Improved seeds or poultry	Once at outset	Households	AEW
	Demonstration gardens/farmer training centres	Once/2 months	Community	District agriculture officer and AEW
	Cooking demonstrations	Once/2 months	Community	HEW and AEW
Systems Strengthening	Integrated training	Routine	HEW/AEW	Kebele
	Integrated monitoring	Routine	HEW/AEW	Kebele
Multisectoral Coordination	Multisectoral coordination work planning and reporting	Once/month	Kebele	Region and district

*Health extension worker.

†Infant and young child feeding.

‡Agriculture extension worker.

SURE, Sustainable Undernutrition Reduction in Ethiopia.

Enhanced nutrition services include:

Counselling of mother–father pairs delivered jointly by teams of one HEW and one AEW. By jointly targeting mothers and fathers, the intervention focuses first on negotiating improved nutrition and agricultural behaviours and second on building awareness of gender-based roles and interactions between men and women within the household to facilitate improved child feeding practices.

Women’s and men’s group dialogues in community networks of 30 households led by a HEW or AEW, respectively, to negotiate community actions to improve dietary diversity and child feeding. To influence community norms, all women and men resident in the network are invited to participate.

Health and agriculture development army leaders (a woman or man, respectively) selected from one of each six households to support government policy conduct routine meetings with the remaining five women or men in the network. Following an abbreviated local SURE training, the leaders support dissemination of IYCF and agricultural messages during these meetings.

Cooking demonstrations on how to prepare complementary foods are delivered jointly by HEWs and AEWs to all members of a 30-household network.

AEWs grow demonstration gardens in communities with emphasis on dietary diversity and nutrition-sensitive agriculture practices, and they train skills at farmer training centres.

A one-time input of improved seeds or poultry is provided to the lowest socioeconomic decile among households with children under 24 months at the start of programme implementation. AEWs advise each recipient on care of inputs and use of the resulting foods produced.

Media messages to promote IYCF practices and dietary diversity are broadcast nationally by radio to districts and regions.

The enhanced nutrition services use a range of social and behaviour change communication tools. The release of global recommendations for IYCF practices by WHO has enabled the development of locally adaptable evidence-based counselling packages for behaviour change interventions in low and middle income countries around the world.9 The standard Unicef training package focuses on provision of knowledge and skills to community healthcare workers to negotiate evidence-based child feeding.23 The SURE training packaged used the IYCF counselling process (3As—Assess, Analyse, Act) developed by Unicef and further adapted the same techniques for application to nutrition-sensitive agriculture advising.

Unicef’s global IYCF counselling package also supplies tested, locally adaptable pictorial tools to support message delivery.23 These tools were previously adapted and subsequently refined for use in Ethiopia by the Alive & Thrive project and the Federal Ministry of Health (FMOH) and adopted for use by SURE.24 The SURE programme then developed an original set of pictorial tools to support agricultural messages, including diversity of crop selection, land and water management, livestock and poultry, hygiene and women’s empowerment. In addition, a locally adapted seasonal food production calendar and a food group poster encouraging daily consumption of four of six food groups were developed to assist household and community-based programme activities.

Systems strengthening and multisectoral coordination components include:

The 5-day integrated SURE training provided at the outset of programme roll-out to all HEWs and AEWs in each district. The training includes an overview of nutrition, dietary diversity, IYCF practices, nutrition-sensitive agriculture practices and facilitation and negotiation skills. For many AEWs, SURE is the first nutrition training received.

An integrated monitoring system designed to feed upward into reports compiled by the multisectoral coordination committees and to facilitate joint health and agriculture sector work planning and supportive supervision at kebele level.

Financial support and oversight of the routine function of kebele and district multisectoral coordination committees, including regular meetings to review SURE and other nutrition activities comprising the National Nutrition Programme. Members are orientated to committee roles and responsibilities under federal guidelines at the outset of the programme.

Implementation status

SURE is focused on selected districts in Ethiopia’s four agrarian regions (Amhara, Oromia, Tigray and SNNP). There are three phases of programme implementation planned. The first phase rolled out the SURE programme to 63 districts between February and July 2017 and the programme presently reaches an estimated 800 000 children under 24 months. The second and third phases of programme implementation will include an additional 87 districts in total and are planned for 2018 and 2019, respectively. This study protocol pertains only to districts included in the first phase of programme implementation.

Study status

The baseline survey was completed in July 2016. The first round of process evaluation data collection was conducted in June 2017 and the second round in December 2017. A third round of process study data collection is planned in 2018, to be followed by the endline survey in 2019. This paper provides the protocol for the overall programme impact evaluation and for the process evaluation studies.

Aim and objectives

The SURE evaluation aims to demonstrate the extent to which complementary feeding and dietary diversity for young children are increased and stunting reduced among SURE intervention districts versus comparison districts. The findings of the evaluation study will be used to inform the Government of Ethiopia of the effectiveness of the programme and to support learning and course correction.

The objectives are:

To evaluate impact by gathering, analysing and synthesising evidence to determine whether SURE results in decreased stunting and increased dietary diversity and MAD among infants and young children.

To evaluate the quality of programme implementation and to inform refinements to intervention design and delivery.

To understand the mechanisms of change by which SURE activities impact key outcomes and nutritional status of children.

To develop a contextual record over time of factors that may have affected implementation, mechanisms of change and outcomes.

Methods and analysis

Programme theory

The programme theory, summarised in figure 1, was developed using the Unicef conceptual framework for undernutrition and agriculture–nutrition impact pathways developed by the International Food Policy Research Institute.25 Programme impact pathways map activities to explain how the intervention is expected to contribute to a chain of results that produce the intended or actual impacts. We theorise that the package of interventions improves child growth and health status by increasing household acquisition of diverse foods and improving child feeding practices, in addition to reducing exposure to pathogens through improved human and animal hygiene and increasing access to child healthcare services. These changes to nutrition and agriculture behaviours may be supported by increased paternal contribution to child diets and care practices. The variety and frequency of programme contacts, including interpersonal counselling, men’s or women’s group dialogues, community events such as cooking demonstrations and media messages, are expected to impact the range of outcomes.

Figure 1

Programme impact pathway. The hypothesised pathways of impact between programme activities, behavioural outcomes and child nutrition status. IYCF, infant and young child feeding

Study design

The SURE study is a quasi-experimental theory-based evaluation using mixed methods. Repeated cross-sectional household surveys conducted at the same time of year at baseline and endline from intervention and comparison districts will be used to determine impact on stunting and children’s diets. Quantitative and qualitative methods will be used to examine processes—including the quality of implementation, mechanisms of change and contextual data—to help interpret findings of the impact evaluation.

Study site and population

FMOH selected 150 districts in the four agrarian regions (Amhara, Oromia, Tigray and SNNPR) to receive the SURE intervention. Of those, 63 were selected by FMOH to receive the intervention in the first of three roll-out phases. The evaluation study area was selected from first-phase districts to provide both the longest duration of intervention and the largest potential difference between intervention and comparison groups. The intervention targets primarily households with children under 24 months. Community participants and health and agriculture staff are also targeted.

Impact evaluation

Main hypotheses and outcome indicators

Primary outcomes include stunting and MAD. The main study hypothesis is that complete and sustained programme implementation over a minimum period of between 24 and 28 months will lead to at least:

A 0.21 increase in mean height-for-age z score in children aged 24–47 months.

A 10% reduction in stunting in children aged 24–47 months.

An increase of 0.17 in the number of food groups consumed by children 6–23 months.

A 6% increase in children 6–23 months consuming the MAD.

Children in the 24–47 months age range will be assessed for stunting outcomes. Growth faltering is most pronounced from 0 to 23 months of age.26 Assessment of stunting in children after this period will allow for the longest duration of exposure to the intervention (fully 24 months).

A range of other behavioural outcomes and programme coverage indicators will also be measured to assess programme impact pathways (see table 2).

Table 2

Table of primary and secondary outcomes

	Outcome	Indicator
Nutrition impacts	Stunting*	Proportion of children 24–47 months with height-for-age z score <−2.0
	Wasting	Proportion of children 24–47 months with weight-for-height z score <−2.0
	Anaemia	Proportion of children 6–47 months with Hb measurements <11.0 g/dL
Longer term behavioural outcomes	Minimum acceptable diet (MAD)*	Proportion of children 6–23 months with MAD†
	Minimum dietary diversity (MDD)	Proportion of children 6–23 months with MDD†
	Exclusive breastfeeding	Proportion of children 0–5 months breastfeeding and not consuming any other liquids or food
	Timely initiation of complementary feeding	Proportion of children 6–8 months consuming complementary foods
	Household food security	Proportion of food secure households (HFIAS)‡
	Women’s minimum dietary diversity (W-MDD)	Proportion of women with minimum dietary diversity (MDD-W)§
Mid-term behavioural outcomes	Household food production and consumption	Mean number of food groups produced and consumed
	Women’s empowerment	Proportion of mothers with decision-making input in the household
	Paternal support for child care	Proportion of husbands supporting child care or household chores
	Hand washing	Proportion of households with water and soap present for hand washing
Programme coverage	IYCF counselling coverage	Proportion of caregivers receiving a joint visit for nutrition counselling in the previous 2 months
	Coverage of community-based nutrition services	Proportion of mothers attending GMP in the community in the previous 1 month

*Primary outcomes.

†WHO/Unicef infant and young child feeding (IYCF) standard indicators.

‡Food and Nutrition Technical Assistance Project (FANTA) Household Food Insecurity Access Scale (HFIAS) indicator.

§FAO Minimum Dietary Diversity-Women (MDD-W) indicator.

Hb, haemoglobin.

GMP, growth monitoring and promotion.

Sample size and selection

The sample size calculations for the household survey are based on detecting a change at endline in nutritional status and key IYCF practices that can be attributed to the intervention. Detectable differences in key IYCF indicators for intracluster correlation coefficients of 0.03 and 0.08 with 80% power with a significance level of 5% are shown in table 3.

Table 3

Detectable difference by indicator and estimated baseline correlations

24–47 months	Baseline	Detectable difference (ICC*=0.03; DE† 2.8)	Detectable difference (ICC*=0.08; DE† 5.7)
Stunting	40%	7%	10%
Stunting (24–47 months) allowing for baseline correlations of 0.5–0.8	40%	6–4%	9–6%
HAZ (24–47 months)	mean; −1.5	0.15	0.21
HAZ (24–47 months) allowing for baseline correlations of 0.5–0.8	mean; −1.5	0.13–0.09	0.18–0.13

*Intracluster correlation coefficient.

†Design effect.

HAZ, height-for-age z score.

Sample sizes at baseline and endline are presented in table 4. Sample sizes are based on a fixed number of kebeles per district (four at baseline and six at endline—the smaller baseline sample size was due to a shortened data collection window) and a fixed number of households (15) per kebele. Based on data summarised from the baseline survey completed in July 2016, we estimate that 15% of households will have two eligible children 0–47 months. The number of children we expect to sample at endline is shown in table 4.

Table 4

Actual baseline sample sizes achieved and expected sample sizes for endline

Survey	Districts	Kebeles	Households	Children
Baseline	72	288	4299	761 children 0–5 months 1848 children 6–23 months 2371 children 24–47 months Total 0–47 months children sampled=4980
Endline	72	432	6480	932 children 0–5 months 2796 children 6–23 months 3728 children 24–47 months Total 0–47 months children estimated ≈ 7456

Prior to the baseline survey, 36 intervention districts were randomly selected within four regional strata with the number per strata determined in roughly equal proportions to the 50 first-phase intervention districts. Accordingly, 18 districts were selected from Oromia, 8 from Amhara, 6 from SNNP and 4 from Tigray regions. Random selection was done in Stata 14. Comparison districts were selected in equal number per the four regional strata to those selected among the intervention districts (see figure 2). Districts were roughly matched based on stunting prevalence tertiles (low, medium or high based on district-level small area estimates provided by the World Bank27) and the presence of the Productive Safety Net Programme as a proxy for food insecurity. Districts were not individually matched but selected to provide for a reasonable balance of characteristics between groups. The same intervention and endline districts selected at baseline will be sampled again at endline.

Figure 2

Map of study sites in the four agrarian regions of Ethiopia.

Kebeles selected for both baseline (4) and endline (6) were selected at study outset using probability proportional to size sampling from lists and population data provided by district officials. Selections were completed in Stata 14. Each was completed independently, producing different kebele samples for baseline and endline. At baseline, gotes (sub-kebeles) were next selected by simple random sampling conducted by team supervisors on arrival to the kebele during data collection (using the paper in hat method). Within the gote catchment area, a complete listing of all households with children under 24 months was then conducted by data collectors and 15 were selected using systematic random sampling. At endline, the same gote selection procedures will be used.

At baseline, all resident children 0–47 months within a selected household and their respective mothers were listed in the following age groups: 0–5 months, 6–23 months and 24–47 months. Where only one child for any/all age categories was present, all eligible children were selected (up to three children). Where multiple children from a single age category were present, one child was randomly selected per category by the computer-assisted personal interview survey programme. The mother of each selected child was also selected for interview. The same procedures will be used at endline.

Data collection tools

The baseline and endline household questionnaires comprise eight modules (see online supplementary file 1). Household-level modules comprise basic characteristics, household food security and dietary diversity, agriculture practices including food production and consumption and women’s empowerment. A module on IYCF knowledge, behaviours, child care practices and intervention services is directed to the mother of the child 0–23 months. A module on primary health and agriculture services targeted to households with children 6–47 months is directed to the mother of the child 6–47 months. The final module comprises anthropometry and finger prick blood samples for estimation of haemoglobin concentration of the child(ren) 6–47 months.

The household survey questionnaires were developed to conduct face-to-face interviews with household members. Questionnaires were translated into Amharic, Oromifa and Tigrigna, back-translated into English and field-tested prior to baseline survey deployment.

Data analysis

Impact on stunting and MAD in addition to other secondary outcomes will be estimated through mixed effects models with a random effect for cluster (kebele) to account for correlations among women and children in the same cluster. Fixed effects will be area (intervention vs comparison), time (follow-up vs baseline) and the interaction between area and time. The key effect of interest is the interaction, which estimates the change from baseline to follow-up in the intervention area relative to change in the control area (analogous to a difference in differences approach). The primary statistical analysis will be carried out at the individual level and will account for clustering.

Following the baseline survey, the following analysis procedures were used: indicators (and other factors) were tabulated by area (intervention/comparison). No significance tests were performed to test for differences. Descriptive statistics for continuous variables included the mean, SD, median, range and the number of observations. Categorical variables were presented as numbers and percentages. Missing data did not exceed that allowed for as data were monitored for completeness during the survey and followed up at the time of collection.

At future follow-up, the results and differences from baseline data will be tabulated by area and presented with the estimated effect of the SURE programme and 95% CI. A confirmatory cluster level analysis will also be carried out. All analyses will adjust for confounders, chosen from among contextual factors that change over time, by assessing whether the adjustment changes the effect estimate by 15% or more. Missing data will be identified by a data manager and followed up at the time of data collection. Unless there is considerably more missing data than that allowed for, no formal statistical methods will be used to account for missing data in the analysis.

Causal mediation analyses will be explored as part of the process evaluation to identify mechanisms of change. Structural equation modelling and/or a non-parametric approach based on the potential outcomes framework will be used to explore potential prehypothesised mediators of any observed intervention effects.

Process evaluation

Mixed methods will be used to evaluate programme processes. Outcome indicators from the endline survey will be used to assess programme reach and behavioural mechanisms of change. Qualitative studies will contribute data on the fidelity, dose and effectiveness of programme activities in intervention areas and drivers of behaviour change in intervention and comparison areas. Overall process evaluation outcomes, research questions and methods are summarised in tables 5 and 6.

Four qualitative studies will contribute data in response to questions on fidelity, dose and effectiveness:

Fidelity and dose of the SURE programme training delivered to HEWs and AEWs.

Fidelity and dose of the SURE programme delivered to orientate multisectoral coordination teams at district and kebele levels.

Function and effectiveness of local multisectoral nutrition coordination teams to support delivery of SURE programme activities.

Fidelity and dose of the SURE programme delivered to mothers and fathers of children under 2 years.

Three qualitative studies will assist to evaluate behavioural changes and mechanisms of programme impact:

Barriers and facilitators affecting adoption of recommended IYCF practices and diverse food consumption.

Barriers and facilitators affecting adoption of nutrition-sensitive agricultural practices and diverse food production.

Role of gender and social norms in the delivery and practice of IYCF and nutrition-sensitive agriculture messages.

Samples will be drawn purposively from preselected endline districts and kebeles from all four regions of programme implementation. To ensure that a broad range of implementation areas are sampled, the number of districts selected for each qualitative study will comprise: Oromia (n=3), Amhara (n=2), Tigray (n=2) and SNNP (n=2). For the three process studies investigating behavioural change and mechanisms of impact, comparison districts will also be sampled to capture contextual factors—for example similar nutrition interventions—as follows: Oromia (n=1), Amhara (n=1), Tigray (n=1) and SNNP (n=1). The number of kebele sites and the total number of observations, interviews and focus group discussions conducted for each study will depend on reaching the theoretical saturation point among participants with a particular set of characteristics. To ensure heterogeneity, a diverse set of eligible participants will be recruited for each study based on known or hypothesised cultural, socioeconomic characteristics or other factors.

Data collection tools and analysis

Appropriate topic guides and tools for each study and method will be developed and tested prior to field deployment. Framework analysis will be used to analyse qualitative data and Nvivo V.11 will be used to code and construct thematic areas from the collected data. Further analyses will be carried out to generate conceptual models or theories that might potentially explain change or absence of change in key behavioural indicators.

Patient and public involvement

Patients were not involved in designing the study or setting the primary research outcomes. Patients will be invited to contribute to the evaluation dissemination strategy.

Ethics and dissemination

The baseline protocol was approved by the scientific and ethical review committee (SERC) at Ethiopian Public Health Institute (Ref number: SERO-54-3-2016) and by the London School of Hygiene and Tropical Medicine (LSHTM) ethics committee (Ref number: 10937) prior to data collection. Process evaluation studies have been approved by SERC (Ref number: SERO-047-01-2017) and study approval processes are ongoing. The full protocol has been approved by the LSHTM ethics committee (Ref number: 12268). By approval of both institutional review boards, informed oral consent is given by caregivers of young children and witnessed by the local HEW as a result of the known reticence of illiterate persons in Ethiopia to sign or thumbprint unfamiliar documents. Written informed consent is obtained from all other study participants.

The findings of the implementation process evaluation will be disseminated to partners, including the FMOH and the Federal Ministry of Agriculture and Natural Resources and regional representatives, through a consultative workshop to facilitate programme course correction. A final dissemination meeting will also be held to present final results of the evaluation study. Peer-reviewed publications and conference presentations will disseminate results to the broader research community.

Discussion

The potential synergies between agriculture and nutrition are well recognised,10 28 and evidence of multisectoral programme impact is beginning to emerge.29 30 Yet evidence of the ‘how’ and ‘why’ underlying the effectiveness of complex nutrition interventions remains sparse.10 This paper describes the design for impact and process evaluation of a multisectoral nutrition education intervention linking nutrition-sensitive agricultural practices to improved complementary feeding. The results of the evaluation will also provide evidence on drivers of diet, agriculture and child feeding-related behaviours in the context of government-led behaviour change intervention at scale.

Table 5

Implementation evaluation outcomes, questions and methods

Assessment domain		Questions related to SURE	Data collection methods
Fidelity	A.1.	How much of the SURE training was delivered?	Observation, KII*
	A.2.	Did HEWs/AEWs† retain knowledge and skills as intended?	Observation, KII, FGD‡
	A.3	Were mothers/fathers asked to commit to household plan of action and were plans followed through?	Observation, KII
	A.4	Were media messages broadcast and tablet-based messages deployed as intended?	KII
	A.5.	Were district and kebele nutrition coordination teams orientated to their duties and provided tools and budgets as intended?	FGD
	A.6	Did coordination teams meet and function as intended?	Observation, KII, FGD
	A.7	Was SURE integrated in planning, implementation and monitoring of health/agriculture sector-specific policy priorities through national to kebele level structures?	Observation, KII, FGD
Reach	B.1.	How many HEWs/AEWs and HDA/ADA were trained?	Endline survey
	B.2	How many households and communities received regular SURE services, including joint HEW/AEW visits, men’s and women’s group dialogues and cooking demonstrations?	Endline survey
	B.3	How many of the target beneficiaries were exposed to the radio and tablet-based media messages?	Endline survey
	B.4	How many district and kebele nutrition coordination teams met as intended?	Endline survey
Dose	C.1.	Which contents and activities of the SURE training were understood and retained best by HEWs/AEWs?	KII, FGD
	C.2.	Which messages related to IYCF§ counselling and nutrition-sensitive agriculture were best delivered and understood and accepted by mothers and fathers?	Observation, KII, FGD
	C.3.	Which media messages were delivered and retained best?	KII
	C.4.	Which tasks did the nutrition coordination teams understand and enact?	Observation, KII
Effectiveness	D.1.	Were proximal outcomes (gains in knowledge and beliefs) achieved as a result of the SURE delivery?	Endline, KII
	D.2.	Do local actors and authorities (nutrition coordination teams) effectively support delivery of the intervention?	KII, FGD
Context	E.1.	What social, economic, political or other environmental factors affected delivery and receipt of the SURE programme?	Routine data, KII, FGD

*Key informant interview.

†Health and agriculture extension workers.

‡Focus group discussion.

§Infant and young child feeding.

SURE, Sustainable Undernutrition Reduction in Ethiopia.

HDA, health development army leader.

ADA, agriculture development army leader.

Table 6

Mechanisms evaluation outcomes, questions and methods

Outcomes		Questions related to SURE	Data collection methods
Intermediate outcomes	F.1.	Did fathers and mothers become more gender aware?	KII*, FGD†
	F.2.	Were fathers motivated to support caring practices? Did they adopt IYCF‡ practices?	KII, FGD
	F.3.	Did other household and community members support improved IYCF practices? Barriers/facilitators?	KII, FGD
	F.4.	Did fathers/mothers adopt improved nutrition-sensitive agriculture practices?	Endline survey, FGD
	F.5.	Did fathers/mothers have access to agriculture inputs?	Endline survey, KII
	F.7.	Did fathers/mothers increase diversity of own production?	Endline, KIIs
	F.8.	Did fathers/mothers adopt improved water, sanitation and hygiene practices?	Endline, FGD, KIIs
Longer term outcomes	G.1.	Did fathers/mothers adopt IYCF practices? What were the barriers/facilitators?	Endline survey, KII, FGD
	G.2.	Did fathers/mothers acquire more nutritious foods for consumption? What were the barriers/facilitators?	Endline survey, KII, FGD
	G.3.	Did fathers/mothers allocate income for nutritious foods or child healthcare and why?	Endline survey, KII, FGD
	G.4.	Were messages on IYCF/dietary diversity diffused within the community?	FGD, KII
Context	E.1.	What social, economic, political or other environmental factors affected child feeding and dietary diversity behaviours?	Routine data, KII, FGDs

*Key informant interview.

†Focus group discussion.

‡Infant and young child feeding.

SURE, Sustainable Undernutrition Reduction in Ethiopia.