An Iterative Process for Training Design and Implementation Increased Health Workers' Knowledge for Taking Nutrition Behavior Change to Scale.

BACKGROUND: The shortage of skilled, motivated, and well-supported health workers is a major barrier to scaling up nutrition interventions and services.
OBJECTIVE: The objective of this study is to describe the process for developing and implementing a training of health personnel for the delivery of the Integrated Strategy for Attention to Nutrition (EsIAN), an evidence-based strategy for promoting infant and young child feeding through primary health care in Mexico. The specific objective is to provide a case study and highlight challenges, as well as elements to successfully mitigate these, and discuss potential applications of findings beyond the Mexican context.
METHODS: The design and implementation of training followed a 5-phase process: situation analysis, formative research, large-scale feasibility study, redesign and scale up, and evaluation. We conducted document reviews, surveys, and focus groups during the first phases to inform and refine the training, as well as a pre- and posttraining telephone survey to evaluate change in knowledge.
RESULTS: The initial phases of the design provided a clear understanding of the opportunities and challenges for promoting infant and young child feeding, as well as health workers' routines and practices, which informed training design. The feasibility study allowed tailoring and refinement of training. The vertical coherence and coordination between the federal and state levels during redesign and scale up facilitated compliance with training timeline and process. Evaluation results showed significant improvement in knowledge posttraining of up to 19 percentage points.
CONCLUSIONS: The EsIAN training component for health providers was developed using a systematic approach to consolidate and generate relevant evidence, following an iterative process to test, learn, and improve both design and implementation. This process allowed for flexibility to take advantage of new opportunities and respond to findings from iterations. Garnering and ensuring political support allowed for continuity and sustainability of actions.

Introduction

The prevention of the double burden of malnutrition and promotion of healthy growth are global health priorities (1–3). Benefits of preventing malnutrition in all its forms are well documented and extend over the life course. They range from decreased mortality and improved cognitive development in infancy to improved school performance for children and increased work capacity in adulthood (3–5).

Targeting the first 1000 d, from conception to the first 2 y of life, with effective nutrition-specific interventions such as the promotion of adequate infant and young child feeding (IYCF) is recommended to prevent malnutrition (6, 7). A growing body of evidence also suggests that adequate nutrition during this period confers long-term benefit for the prevention of noncommunicable diseases in later life (4, 8, 9).

In many settings, regular visits to a health facility for preventive (well-child) and sick care of infants and young children are promoted during the first 1000 d. During these visits, caregivers of infants and young children routinely interact with health workers such as nurses, doctors, and health promoters, who, in many contexts, are credible and reliable sources of information and advice (10, 11). Health workers are thus well positioned for promoting adequate IYCF and providing preventive counseling.

Despite this potential, health workers in some settings receive little nutrition training and are ill-equipped to assess diets and nutritional status as well as provide effective nutrition counseling (12, 13). The shortage of skilled, motivated, and well-supported health workers is a major barrier to scaling up nutrition interventions and services (14). This limits the capacity to implement effective nutrition services and long-term solutions to promote adequate nutrition.

The existing literature does not provide clear direction on the capacity that health workers require for large-scale nutrition programs. Documentation is particularly weak on gaps in delivery systems and how interventions can address them (15), leading to lack of practical, field-tested frameworks, processes, and tools that can be applied in diverse contexts to improve IYCF (16). The studies included in this supplement identified several of these capacity gaps and resulted in the identification of both challenges to addressing those and potential avenues to do so.

In this article, we describe the process for developing a training of health personnel for the delivery of the Integrated Strategy for Attention to Nutrition (referred to as EsIAN), an evidence-based strategy for promoting IYCF through primary health care (PHC) in Mexico. The specific objective of the study is to provide a case study and highlight challenges, as well as elements to successfully mitigate these, and discuss potential applications of findings beyond the Mexican context.

EsIAN is a national strategy to strengthen the health and nutritional component of the conditional cash transfer program PROSPERA (referred to as CCT-POP). It aims to address nutrition in all its forms through a life course perspective and improve the health and nutrition of its beneficiaries (17). EsIAN uses strategies for which there is proven efficacy and effectiveness, with a focus on the first 1000 d of life, the evidence for which has been presented in previous articles in this supplement (18). It has 3 main components: 1) training for health providers and an integrated behavior change communication (BCC) strategy to promote adequate nutrition practices; 2) the provision of nutritional supplements for pregnant and lactating women (PLW) and children from 6 mo to 5 y of age, with a differential scheme for urban and rural areas; and 3) procurement of functioning equipment for nutritional assessment (height, weight, and anemia) and BCC (television and DVD player).

The training aims to ensure that the health workers have the knowledge and skills necessary to adequately promote 1) appropriate weight gain, the use of multiple micronutrient tablets, and healthy eating during pregnancy; 2) breastfeeding and complementary feeding practices according to current recommendations in children 0–23 mo of age, including the use of nutritional supplements starting from 6 mo of age (varying by urban and rural areas); and 3) healthy eating for children 2–5 y of age, including the use of micronutrient powders.

Methods

The design and implementation of training followed a 5-phase process: situation analysis, formative research, large-scale feasibility study, redesign and scale up, and evaluation. During the first 4 phases, we used various data collection methods to inform and refine the training and supervision (), which were reviewed and approved by the Research, Ethics and Biosecurity Commissions of the National Institute of Public Health (INSP) in Mexico. The evaluation was approved by the Research Ethics Committee of the Ibero-American University in Mexico.

TABLE 1

Data collection methods[1]

Method	Description
Situation analysis
Review of journal articles, gray literature, program reports, and organizational norms	Revision of the current models, methods, and approaches used for training health personnel Revision of the organizational structure and internal procedures of IMSS-PROSPERA, Secretary of Health, and Sedesol
Formative research 50 semistructured interviews with health workers 32 structured observations of consultation for children <24 mo 8 focus groups with health workers	Assess health workers’ knowledge, attitudes, and practices related to promotion and counseling on IYCF and growth monitoring. Identify barriers to effective promotion at PHC. Understand organization of PHC, including provision of counseling services, roles and responsibilities of personnel, and training and supervision systems.
Feasibility study
Cross-sectional baseline and endline survey with 243 nurses, physicians, and health promoters 16 focus groups with nurses, physicians, health promoters, and nurse technicians/community volunteers (vocales) 4 in-depth interviews with health personnel at the first level of the training cascade	Assess changes in knowledge and perceptions of trainees on key EsIAN topics after the training. Assess health workers’ experience with the training, including key strengths and weaknesses of the training and recommendations for improvement. Assess experience with providing training to next level of the cascade.
Redesign and scale up
Assessment of learning abilities and technological capabilities Pre- and posttests Satisfaction survey for 851 trainees (level 1) and 94,026 trainees (level 2)	Assess trainees’ learning abilities and technological capabilities before training. Assess changes in knowledge and perceptions of trainees on key EsIAN topics after the training. Assess health providers’ experience with the training, including if they perceived it as useful and their recommendations for improvement.
Evaluation
Telephone survey with 1586 respondents at baseline and 796 at postintervention	Assess association between training and changes in knowledge of health workers.

EsIAN, Integrated Strategy for Attention to Nutrition; IMSS, Mexican Institute of Social Security; IYCF, infant and young child feeding; PHC, primary health care.

The aim of the situation analysis was 2-fold: 1) to review the literature covering current theories, models, methods, and approaches used for training and supervision of health personnel and 2) to gain an understanding of the structure and functioning of organizations involved with the delivery of health services to CCT-POP beneficiaries [State Health Services, Mexican Institute of Social Security (IMSS)–PROSPERA, Secretary of Health, National Coordination of PROSPERA, Secretary of Social Development (Sedesesol)], including their operational norms, management, and roles and responsibilities. This phase was instrumental for identifying knowledge gaps, particularly about the organizations’ processes and structures, that were then addressed during the formative research. The formative research used mixed methods to assess health workers’ current practices and barriers to promote healthy pregnancy, adequate IYCF practices, and healthy growth in urban and rural areas, including both indigenous and nonindigenous populations in rural areas [for more information, see Bonvecchio et al. (18) in this supplement]. We explored barriers and opportunities to effective counseling at PHC, as well as limitations for quality interactions with PLW and caregivers and potential opportunities for change. Data were collected from 56 PHC centers, including 39 urban (>15,000 inhabitants) and 17 rural communities (<15,000 inhabitants), randomly selected from the state of Puebla. Physicians, nurses, health promotors, health assistants of IMSS-PROSPERA and Secretary of Health, and CCT-POP community health volunteers participated in the research.

We consolidated relevant information from several previous studies with caregivers and community members [for more information, see Theodore et al. (19) in this supplement]. This review, together with the formative research results, provided an in-depth understanding of the common IYCF practices, barriers, and opportunities for change and was used to develop the content and structure of EsIAN's training and BCC strategy.

Between 2008 and 2012, EsIAN's feasibility study ran in 91 PHC centers in the states of Puebla, Michoacan, Guerrero, and the State of Mexico in central-southern Mexico. The program was implemented by the staff of the health centers, as per design. Procurement and distribution of the supplements, BCC materials, and equipment for the EsIAN were implemented and fully managed following the procedures of the Secretary of Health. The research team (AB) led the training of the health professionals in collaboration with the health sector, as described below. The primary objective of this study was to test EsIAN's feasibility and its impact on health workers’ knowledge, attitudes, and practices, as well as the beneficiaries’ knowledge and practices, with the ultimate goal of providing key recommendations for improvement. Cross-sectional baseline and endline surveys with nurses and physicians, as well as focus groups and interviews with health personnel, were conducted.

EsIAN's scale-up began in April 2014, with the implementation of a nationwide, in-person training of health workers in the state of Guanajuato, followed by the launch of the virtual training. Throughout the training, health workers were able to request technical support via email and completed a baseline survey to assess learning profiles and technological skills and a satisfaction survey at the end of the training. The first set of health workers completed training in August 2014.

The evaluation was launched in 2016 to assess the influence of the training on knowledge among providers in the states of Chihuahua, Oaxaca, and Veracruz. The implementation of EsIAN in these states was purposively delayed until September 2016. Baseline data were collected in July 2016 and posttraining data between March and April 2017. Two independent direct random samples were drawn in each period, aiming at assessing aggregate changes. Sampling frame included 10,814 health workers expected to be trained at baseline and 14,449 at posttraining. Sample size was estimated to detect small changes in test scores (2–3 points) stratified by type of provider, assuming 95% power. It is representative of CCT-POP health workers in these states.

Using a database of CCT-POP health workers provided by the Secretary of Health, at baseline and posttraining, participants were identified through a direct random sampling method, stratified by state, type of provider (i.e., physicians, nurses, or nurse technicians), and institution (i.e., state health clinics or IMSS-PROSPERA). Any provider working full-time in a primary care center serving the program's beneficiaries was eligible for inclusion. A telephone survey, pretested with 59 health providers, was conducted at baseline and posttraining. Trained interviewers administered the survey, which lasted approximately 20 min and was answered by 1586 respondents at baseline and 796 at postintervention. To ensure data collection quality, a subsample of interviews was recorded and analyzed to assess errors in respondents’ eligibility, biases in procedures of data collection, and missing data and data patterns by the interviewer. Participation rates were 47.3% and 41.5% at baseline and postintervention, respectively.

The survey included 4 different questionnaires relating to the training materials and contents of EsIAN (20, 21). The first questionnaire, administered to all participants, addressed general knowledge about nutrition and supplementation and the implementation of EsIAN. The remaining 3 questionnaires covered topics related to pregnancy, nutrition for children under 5 y of age, and breastfeeding. Only 1 of these 3 questionnaires was randomly administered to each respondent to ensure survey duration of 20 min or less.

We calculated descriptive statistics to compare the study samples and to provide postintervention indicators of implementation. Bivariate analyses were performed for the 10 items that measured knowledge at baseline and postintervention in the first questionnaire. In addition, to assess the total influence of EsIAN training in knowledge, we performed a propensity score matching (PSM) analysis. This quasi-experimental technique allows matching participants with similar characteristics. We followed a 2-step statistical approach. First, using a logit model, we estimated the probability of being in baseline or postintervention given a vector of covariates (i.e., age, sex, type of provider, institution, state). This generated a propensity score p(X) that allowed identifying an intervention group and a control group. In the second stage, the average effect on the treated was computed. All analyses were performed using STATA v.15 (StataCorp, 2017). All P values were 2-tailed, and the statistical significance was set at P < 0.05.

Results

The phases of the design process provided an understanding of the opportunities and challenges for promoting IYCF as part of the health workers’ routines and practices in PHC. Its iterative nature allowed us to tailor and refine the training design.  summarizes the learnings and implications for design.

TABLE 2

Formative research results and implications for training design[1]

Phase	Key finding	Implication for training design
Situation analysis/formative research	The health institutions have complex structures and hierarchies.	Design of a 4-level cascade training for physicians and supervisors of IMSS delegations and state-level responsible for the health component of CCT-POP at the Secretary of Health (level 1); IMSS physicians with role of supervisors, supervisors of community actions, and coordinators of health education at regional level (level 2); physicians, nurses, and health promoters at IMSS local level and physicians, nurses, nurse technicians, and health promoters of the local Secretary of Health (level 3); volunteer health promoters, midwives, and rural health assistants of IMSS and health assistants and midwives of the Secretary of Health and community health volunteers managed by Sedesol (level 4).
Situation analysis/formative research	The Secretary of Health and IMSS have different structures, personnel, roles, and ways of working. Both are state led and thus function differently across states.	Design a flexible training that could be adapted to each organization in the different states. Raise awareness among high-level decision makers about the importance of EsIAN.
Situation analysis/formative research	Health personnel have different roles, and there is a strong hierarchy within health services.	Conduct separate, in-person training for different types of personnel. Promote physicians who train physicians and nurses who train nurses.
Formative research	Health personnel counsel on nutrition but in an unstandardized way and providing inconsistent messages.	Develop a simple counseling model that can be used by physicians and nurses to counsel program beneficiaries. Develop simple, relevant, and clear messages that could be used as part of the counseling model. Promote consistent use of messages, making sure that the same messages are used by all the different types of personnel.
Formative research	CCT-POP communication materials (such as flipcharts) used for group counseling provide inconsistent messages, some of them outdated.	Recommend revision and updating of messages to CCT-POP, to be consistent with international guidelines and EsIAN's recommendations.
Formative research	There are technical norms that address some of EsIAN's key nutrition topics, but some of them are outdated and/or not used by health personnel.	Review technical norms and make reference to them. Identify outdated content in the technical norms and advocate for change.
Formative research	Community health volunteers are motivated to provide nutrition counseling but have low literacy levels.	Develop a training for low-literacy groups.
Formative research	The Mexican population is diverse and has different traditions and customs across areas (urban, rural, and indigenous) and states.	Promote diversity and tolerance of traditions and local customs and context, creating messages that could be adapted to local conditions and resonate with the target audience.
Formative research/feasibility study	Health personnel are motivated to provide nutrition counseling but have limited time to do so.	Develop a simple counseling model that can be used by physicians and nurses to counsel program beneficiaries. Diversify the moments and personnel that provide nutrition counseling. Prioritize key messages that could be used during counseling.
Formative research/feasibility study	Health personnel lack counseling skills.	Develop a simple counseling model that can be used by physicians and nurses to counsel program beneficiaries. Include an in-person component of the training that could be used to train and practice counseling skills.
Formative research/feasibility study/redesign and scale up	There is a high rotation of personnel each year, especially of physicians.	Develop a simple cascade training that can be adopted and replicated by health personnel. Raise awareness among high-level decision makers about the importance of EsIAN to ensure sufficient resources and commitment for its continuous implementation.
Formative research/feasibility study	Health personnel lack technical knowledge about key nutrition topics.	Identify and prioritize key nutrition topics that are most relevant for EsIAN. Identify and address knowledge gaps about these key nutrition topics.
Feasibility study	Health personnel found training difficult to replicate.	Simplify topics and information included in the training. Simplify cascade training.
Redesign and scale up	Health personnel had basic technological skills and were able to take the virtual course.	Continue implementation of virtual training.
Redesign and scale up	Health personnel in level 1 failed the final exam and had to retake the training.	Design and implement a replacement course.

CCT-POP, conditional cash transfer program PROSPERA; EsIAN, Integrated Strategy for Attention to Nutrition; IMSS, Mexican Institute of Social Security.

Situation analysis

We reviewed a series of behavioral change theories and models that provided a theoretical foundation on determinants of behaviors and structured frameworks for developing and managing public health interventions [for more information, see Bonvecchio et al. (18) in this supplement]. The revision of the institutional norms, on the other hand, allowed us to map the type of actors and hierarchies within the health services. The complexity of the organizational structure, hierarchies, and types of personnel identified the need to develop a cascade training to reach the different organizational levels. On the basis of these findings, we identified the need for 4 levels of the cascade within the training program ().

Formative research

The formative research results pinpointed problems and opportunities that informed the training design. As described in Theodore et al. (19), gaps in caregivers’ knowledge and practices provided insights into the content needed within the BCC strategy. For example, contrary to the World Health Organization recommendation of exclusive breastfeeding to 6 mo of age, liquids were introduced early, including water and common remedies (e.g., fresh leaf tea) to relieve infants with colic.

An unanticipated result of the formative research was identifying the existence of important gaps in the understanding of nutrition during the first 1000 d among health workers at all levels. For example, they were unclear on the definition of exclusive breastfeeding and recommendation of age-appropriate complementary foods. Physicians and nurses did not conduct nutrition assessment of infants and young children described in their institutional procedures, focused on assessing weight (and not height) of children, and relied on single measurements instead of growth trajectory with respect to both weight and height to conduct the nutritional assessment (22). We also found that receptivity and interest among working at all levels to improve the quality of service delivery demonstrated a willingness to participate in training activities.

The formative research thus informed the development of a design roadmap, highlighting key characteristics of an effective training tailored to the context. Mainly, we identified the importance of 1) aiming to improve not only technical nutrition knowledge and attitudes of personnel but also their counseling skills; 2) designing a flexible and adaptable training that could be easily used across organizations and their different levels; and 3) distributing workload across different types of health workers—physicians, nurses, nurse technicians, health promoters, and community health volunteers.

The proposed in-person training aimed to strengthen health workers’ capacity for performing basic key activities as part of the regular services for PLW and caregivers of children younger than 5 y (). In particular, the training promoted the consistent use and dissemination of key messages across all health workers and community health volunteers to ensure optimal exposure to the same messages, team training, and integration. It was designed to improve physicians’ and nurses’ counseling skills, including formulating open-ended questions to identify potential problems, checking questions to ensure caregivers’ understanding of the message, and praising for behavioral intention and practice.

We defined the roles and responsibilities of different cadres of staff and workers, basic and simple training to build knowledge and counseling skills, user-friendly job aids, structured supervision to encourage adherence to counseling guidelines, and feedback on monitoring indicators.

Description of the resulting training component and results of the feasibility study

The designed training consisted of an in-person, 4-level, 5-d cascade training for health professionals (physicians and nurses) and community workers (health promoters, nurse technicians, and community health volunteers). The INSP trained the first and second levels of the cascade, while levels 3 and 4 were planned to be trained by levels 2 and 3, respectively.

In June 2009, the INSP held a training in the State of Mexico for 285 IMSS and Secretary of Health personnel of the 4 states. Due to limited resources, the length of the training was reduced from 5 to 2 d. The training had 3 main components, each designed to address a specific aim. It began with a strong sensitizing component, to raise awareness about the role of the health sector in addressing the double burden of malnutrition, the importance of EsIAN in tackling this problem, and their own responsibility of making sure that EsIAN reach the intended audience. The second and third parts of the trainings were designed to improve technical skills and knowledge of health personnel, reinforce and promote the key messages of EsIAN's communication strategy to be used during counseling, and strengthen counseling and problem-solving skills.

Trainers were exposed to all the activities that they were required to replicate for the next level. Pre- and posttests showed an improvement in knowledge, with differences in adequate response rates between –2% and 96%, with a median of 26% improvement. Rapid assessments were used to receive immediate feedback on training content and activities. This information was used to refine the training content and material. As participants found the training information too complex to replicate, the number of presentations was reduced and the information provided was simplified. By the end of the training, the INSP provided detailed description and materials to replicate the training for levels 3 and 4, with the aim to foster training fidelity and effectiveness. Trainers were to organize small group trainings for health and community workers, with no more of 20 participants each. Total training duration was set to ∼25 h.

Several recommendations were formulated as a result of the feasibility study. Given the diverse target groups of EsIAN (PLW, caregivers of children <6 mo, 6–23 mo, and 2–5 years) and the large number of activities prioritized in coordination with the health authorities, the length of the training proved insufficient to cover these activities in detail. As such, the INSP recommended including an additional 2-d workshop for breastfeeding promotion and counseling strengthening. Furthermore, as the original goal of planning specific training for each of the different actors was unfeasible due to the complexity of the organizational structures and lack of resources, the INSP recommended establishing a national training team to facilitate replication and fidelity of training of levels 3 and 4 and to support the states in completing this activity. Last, it was identified that community health volunteers were trained by Sedesol in their communities and did not have a strong connection with IMSS personnel, thus not proving an adequate channel for their training or supervision. The team suggested promoting and positioning community health volunteers as a key figure with a relevant role that complements and strengthens the promotion of key messages.

The endline survey provided key information on the training of levels 3 and 4. Of the 161 physicians and nurses and 49 health promoters surveyed, 86% and 71% received the training, respectively. Training fidelity was low, with varying length and materials used. Only 26% of physicians and nurses and 17% of health promoters were trained for 3 or more days. Furthermore, adequate supporting materials were referred to be used by 83% of physicians and nurses but only by 11% of health promoters. Despite this, over 70% of health providers considered the EsIAN training useful.

The interview results provided a few insights into some limitations during training implementation. Some trainers considered that they were not adequately equipped to provide training due to lack of skills, time, and resources needed to reach health providers in disperse areas. They also indicated that the amount of information was excessive and did not have enough time to discuss it all. The community health volunteers were not involved as part of the training and did not receive the EsIAN material developed for them.

Redesign and scale up

On the basis of the learnings of the feasibility study, we redesigned several aspects of the training, including its model, content, actors, and systems involved. We simplified the model by merging levels 2 and 3, and we adopted a blended training system consisting of an in-person and virtual training (online for level 1, online or offline for level 2). This model was adopted as a response to the high rotation rate of health personnel and the lack of resources and time to hold a 5-d in-person training session and improve training fidelity. In addition, we developed EsIAN's website, with general information of the strategy, BCC materials, contact information, and training registration and log-in.

The in-person training was designed to provide a sensitization and an introduction to the EsIAN, explain the virtual training, and practice communication abilities promoted by EsIAN and conduct nutritional assessment. In turn, the virtual training aimed to build key nutrition-related competencies. Level 1 was assigned a tutor throughout the virtual training who reviewed assignments, responded to questions or concerns, and made sure that the virtual training milestones were completed on time.

The training content was simplified by reducing the number of topics considering key themes defined in coordination with health authorities based on their relevance for meeting EsIAN's objectives. Management of preeclampsia and gestational diabetes, for example, was excluded to prioritize topics such as assessing and promoting adequate weight gain during pregnancy. We also strengthened the counseling component by providing a more structured, step-by-step counseling model for providers. In addition, to improve its quality, it was redesigned to be a flexible, competency-based training with design methodologies and techniques that motivate learning and stimulate interest. For example, it incorporated the use of game design elements (i.e., gamification) and activities that enabled trainee-trainee and trainee-mentor interaction to reduce trainees’ sense of isolation. Tools to facilitate and support the trainee such as autoevaluation tools, case studies, references, help services, and communication tools such as a video about the nutrition supplements were also included. As an incentive, the trainees who passed the course received a certificate provided by the INSP with the authorization of the Secretary of Health and recognized by the Mexican health system.

The federal health authorities requested the appointment of state-level health actors responsible for the implementation of EsIAN at the Secretary of Health and IMSS-PROSPERA. We pursued and promoted the involvement of these new appointees and key decision makers and invited them to a 2-d high-level meeting to sensitize them on the double burden of malnutrition in Mexico and the key role of EsIAN in addressing this problem. The goal of this meeting was to ensure their support and commitment for EsIAN's scale up, which involved securing resources for implementing its training and BCC strategy. In addition, we engaged in more frequent communication with Sedesol, which became a key partner to train the community health volunteers and strengthen their link with the health centers.

Finally, we designed a supportive supervision system to accompany EsIAN's implementation. This system was developed considering the Secretary of Health's monitoring and supervision systems and procedures. The Secretary of Health further adapted and integrated EsIAN's supervision indicators into its systems. This process took a few years but resulted in the sustained quarterly collection of indicators and corrective actions. Training indicators included the percentage of participants who had approved and failed the training during the period.

As of mid-April 2019, EsIAN has trained or was in the process of training 851 and 102,293 health workers in levels 1 and 2, respectively. The first round of training began with a 2-d in-person training of 800 trainers (level 1), followed by a 44-h online course for physicians and nurses and a 35-h online course for health promoters. The course duration was defined in coordination with health authorities and based on the experience of instructional designers developing similar courses. The online course included pre- and posttests, virtual discussions, and a project assignment to design a detailed plan to train 100 level 2 health personnel. Furthermore, the course for levels 1 and 2 included a final exam to assess changes in knowledge and skills, with a passing grade of 70/100. Of the 92,333 health workers in level 2 who took the final exam, 1500 failed the course. Training of level 3 was conducted by Sedesol, using its own training system, which involved holding in-person meetings every 2 mo with community volunteers. These trainings were conducted using materials developed by EsIAN according to Sedesol's requirements.

We faced various caveats during the training of level 1. Some of the health professionals who attended the in-person training were not well suited to train level 2, as they did not provide training as part of their regular activities. Some trainers were reassigned or left their position unexpectedly. In addition, 121 personnel failed the final exam and had to retake the course. This had time and resources implications, as training for some at level 2 had to be delayed.

Some factors facilitated the training process. A fluid and open communication with federal health authorities led to speedy replacement of inadequate trainers and implementation of the replacement course. Health professionals had basic technological skills and did not encounter major challenges to take the course. This was supported by the results of the technological skills exams, in which most trainees demonstrated the skills necessary for completing a virtual training. Continuous support and oversight from tutors ensured that training activities of level 1 were conducted as planned. In addition, monitoring data of online activities facilitated coordination of federal and state authorities to ensure adequate progress and training completion of level 2.

Evaluation

The study sample characteristics mostly did not differ between baseline and posttraining (). The distribution of providers across type of institution, state, and stratum was slightly different between baseline and posttraining. This may be explained by differences in sample size (i.e., the postintervention sample was smaller due to time restrictions).

For the posttraining period, ∼75% of the participants knew about EsIAN. Of those familiar with EsIAN, 58% had received the training materials. These findings suggest a slow diffusion of the intervention and gaps in the implementation process. Despite these challenges, once the providers received the training materials, more than 90% started the training and approved the online training exam.

Providers who did not know EsIAN in the posttraining period were excluded from the bivariate and PSM analysis, as both looked at differences between baseline and postintervention.  shows the bivariate analysis stratified by type of providers. The total score posttraining (i.e., 10 items together) significantly improved for all types of providers; however, the improvement was greater for physicians (6 percentage points) than for registered nurses (5 percentage points) or nurse technicians (3 percentage points). Physicians showed improvements in knowledge areas such as the critical time window for growth and development (14 percentage points), health consequences of the nutritional transition (19 percentage points), and infectious disease as a potential cause of undernutrition (15 percentage points). Significant improvements were also found in nurses and nurse technicians but in a smaller magnitude.

TABLE 3

Percentage of providers who correctly answered items from a general knowledge questionnaire by type of provider[1]

	Primary physicians				Registered nurses				Nurse technicians
Item	Baseline	Post	Difference	P value	Baseline	Post	Difference	P value	Baseline	Post	Difference	P value
Total	61.12	67.36	6.24***	0.00	56.46	61.76	5.30***	0.00	54.70	57.80	3.10**	0.02
1. Causes of nutritional transition and double burden of disease	64.40	71.92	7.52**	0.02	56.67	59.15	2.48	0.60	54.85	55.50	0.65	0.88
2. Main undernutrition issues among children <5 y	57.20	50.52	−6.68	0.06	52.31	52.81	0.50	0.91	44.94	46.15	−1.21	0.77
3. Undernutrition among children <5 y in different regions and population subgroups	75.76	74.03	−1.73	0.57	61.79	59.15	−2.64	0.58	58.65	54.40	−4.25	0.32
4. Critical time window for optimal growth and development	18.98	32.98	14.00***	0.00	13.33	26.76	13.43***	0.00	10.34	16.48	6.14**	0.03
5. Consequences of nutritional transition	31.44	50.87	19.43***	0.00	43.08	49.30	6.22	0.20	47.05	56.04	8.99**	0.04
6. Infectious diseases as a cause of undernutrition among children	56.51	71.58	15.07***	0.00	51.03	66.20	15.17***	0.00	57.17	61.54	4.37	0.31
7. Stunting indicators	83.24	85.61	2.37	0.35	78.21	74.65	−3.56	0.38	69.62	78.02	8.40**	0.03
8. Wasting indicators	59.00	63.85	4.85	0.16	52.56	61.26	8.70	0.07	55.91	64.83	8.92**	0.04
9. Effects of undernutrition in intellectual and labor performance	89.75	90.52	0.77	0.70	89.23	90.84	1.61	0.58	88.61	87.36	−1.25	0.65
10. Links between undernutrition and chronic diseases	74.93	81.75	6.82**	0.02	66.41	77.46	11.05**	0.02	59.92	57.70	−2.22	0.60

Significance level: **P < 0.05, ***P < 0.01.

Because baseline and postintervention were independent random samples, a quasi-experimental technique was used to control for potential unobserved differences between samples.  shows the results from the PSM analysis; the 10 items were collapsed into a single knowledge score. Matching was estimated through different techniques (i.e., kernel, caliper, nearest neighbor) and outcomes were similar. The estimations are based on kernel estimations and show significant improvements in knowledge. Effects were larger for physicians (6 percentage points) than for registered nurses (5 percentage points) or nurse technicians (3 percentage points).

TABLE 4

PSM assessing general knowledge gains between baseline and posttraining by type of provider[1]

ATT 10 items (general knowledge)	Baseline	Post-training	Difference	t Statistics
Physicians	61.28	67.36	6.08***	6.12
Registered nurses	56.47	61.76	5.29***	3.48
Nurse technicians	54.48	57.80	3.32**	2.38

ATT, average effect on the treated; PSM, propensity score matching. Significance level: **P < 0.05, ***P < 0.01.

Discussion

The EsIAN training component was developed using a systematic approach to consolidate and generate relevant evidence, following an iterative process to test, learn, and improve both design and implementation. We gained a clear understanding of health workers’ routines and practices. The design process permitted flexibility to take advantage of new opportunities and respond to findings from the iteration. The uptake of a blended training, for example, took advantage of the available resources and technological skills of health workers while responding to the need to expedite a large-scale, low-cost training in the context of a high rotation of personnel. A timely and coordinated response to the unexpected failure rate of level 1 avoided further delays or problems with scale up.

There is evidence that health workers training to strengthen nutrition counseling services during the first 1000 d can improve the dietary intake and growth of young children (23). Research, however, has commonly been limited to relatively short-term studies in selected populations, and more work is needed to identify the key factors involved in sustaining performance at scale (24, 25). This article contributes to the literature by describing the process for designing a training that has proven effective in improving health workers’ knowledge (26) as part of a national, country-owned, country-led strategy for addressing the double burden of malnutrition in a sustainable fashion.

The integration of programmatic actions into the existing health system was critical for facilitating scale up and sustainability. The integration, however, brought challenges. A careful balance and compromise had to be reached between making technically sound decisions and finalizing and conducting a coordinated scale up of the 3 components of the strategy, including training, according to timeline and available resources. The design process had to adjust to a somewhat slower pace and response from the organizations within the health system and specific resources allocated for EsIAN. For example, monitoring and supervision activities, critical for learning through the scaling-up process, were delayed, and the duration of the in-person training was shortened to respond to reduced resources. The flexibility in the process and actors involved allowed us to adapt to these somewhat unexpected circumstances.

The use of an existing system means understanding and adapting to its processes, resources, and structures. In practice, this translated into a resource-intensive and lengthy process, which was advantageous for the training and overall strategy design as it ensured manageable demands on the health system. Furthermore, the actors involved were aware of and recognized the changing nature of the political, institutional, and policy contexts that governed the design and implementation of the training and overall strategy. Similar to other contexts (27, 28), they deliberately implemented actions to strengthen the enabling environment for ensuring continuity.

Despite its advantages, a similar design process might not be feasible or necessary to replicate under different conditions. A comprehensive and iterative formative research phase was needed to gain an in-depth understanding of the delivery platform, target audience, and behaviors. A shorter, more focused formative research might be sufficient in contexts with less complex platforms and conditions and information on target audience and key behaviors.

Several factors were critical for scale up. High-level political support fostered during the process was instrumental for coordinated action given the multiple organizations and levels involved with the training. A nutrition champion at the Secretary of Health who valued and was familiar with EsIAN and its components helped garner political and financial support, catalyzed action, and provided continuity and direction during government transition. Furthermore, although the health system is decentralized, the vertical coherence and coordination between the federal and state levels facilitated state accountability and compliance with training timeline and process. These factors have also been relevant for scaling up nutrition action in other contexts (29).

The evaluation found that training relates to improved knowledge of health providers, particularly of physicians. These results can be used to maintain support and commitment for implementation of EsIAN and its training component while continuing to tailor and refine the training based on monitoring information. Further research is needed to assess if training improves skills and translates into improved services for mothers and children.

Click here for additional data file.