Building Capacity of Health Professionals in Low- and Middle-Income Countries Through Online Continuing Professional Development in Nutrition.

ABSTRACT: Continuing professional development (CPD) in low- and middle-income countries (LMICs) can build capacity of health professionals in infant and young child feeding (IYCF). However, travel to in-person workshops can be time-consuming and expensive. Thus, we developed a free online course to provide training in IYCF to health professionals globally; the course received overwhelmingly high numbers of registrations. Our aim was to conduct a program evaluation to assess course satisfaction, learning, and application using surveys administered postcourse and 9 months later. Response rates were 99% (n = 835) and 55% (n = 312), respectively. Among those who only partially completed the course, reasons for noncompletion were assessed (response rate 29%, n = 72). Data within a 1-year period were analyzed. Respondents worked in multiple settings and organizations worldwide. Nearly all (99%) reported postcourse that they learned "a lot" or "some" from all topics, and over 70% applied "a lot" of or "some" information 9 months later. In open-ended questions, respondents reported improved knowledge, skills, and competence to conduct their work; they also desired more similar training courses. Many who did not complete the course reported "not enough time" as the main reason (74%), and most (94%) wanted to continue it. The positive response to the course suggests there is an unmet need for CPD for health professionals in LMICs. Our evaluation found that online training was feasible, acceptable, and increased professionals' knowledge and application of IYCF concepts in nutrition programming. Increased use of online CPD offers the potential for global capacity-building in other health-related topics.

Health professionals in low- and middle-income countries (LMICs) face many challenges with participation in continuing professional development (CPD).[1,2] Attending in-person CPD trainings may be time-consuming and expensive, especially for professionals working in remote areas. Providing such trainings may also be too costly if only limited numbers are reached. Nevertheless, these professionals have substantial impact on the health of local populations, thus, their knowledge and skills are critical.

Infant and young child feeding (IYCF) is a key determinant of children's health and, hence, the health of a country's future adult population. The implementation of recommended IYCF practices makes an important contribution to child survival, growth, and development and to the achievement of the World Health Organization (WHO) global nutrition targets[3] and the United Nations (UN) Sustainable Development Goals.[4] A child's first 2 years of life are considered a critical “window of opportunity” to promote health, development, and good nutrition.[5-7] During this period, suboptimal breastfeeding and inappropriate complementary feeding practices put children at risk for undernutrition. This risk is highest for children in the poorest countries in the world, where the combined effects of stunting, wasting, and underweight still produce an enormous disease burden, compared with other developing countries where childhood undernutrition has significantly declined.[8] Nutritionists and other health professionals have a key role in influencing the adoption of recommended IYCF practices in LMICs and beyond, but need CPD to build capacity for programming in this area. Unfortunately, LMICs are seriously challenged by poor-performing primary health care systems, so the delivery of quality health and nutrition services requires investment in capacity-building for health professionals.[9,10]

Here, we describe the need for and development of an online course that provides CPD in IYCF to a global audience. Our aim was to conduct a program evaluation to determine course effectiveness in building capacity and to seek ways to improve the course and future online trainings.

PROBLEM STATEMENT

Building the capacity of health professionals in LMICs in comprehensive IYCF approaches at both the national and local levels is critical to meet global health and nutrition targets.[1,3,11] The United Nations Children's Fund (UNICEF), for example, has developed an IYCF programming guide for health professionals specifically designed to provide information on how to plan, implement, and evaluate evidence-based programs on IYCF for community populations.[12] Traditionally, in-person training has been used to deliver continuing education to nutrition and health professionals in LMICs.[13] UNICEF used an internal program costing exercise based on actual in-person CPD workshops on IYCF and estimated the cost to be approximately $2000 per person. Importantly, these costs are not one-time expenses since ongoing training is needed as guidelines and recommendations change and as staff turn over. Creating more affordable and sustainable methods of providing CPD also aligns with UN agencies' efforts to reduce greenhouse gas emissions through use of virtual-based meetings or trainings.[14]

SOLUTION: ONLINE CPD

The availability of online education has revolutionized the possibilities for capacity-building in public health nutrition, particularly in the global arena.[15] Online training provides a more convenient and affordable way to deliver CPD to nutrition and health professionals in LMICs than in-person training. There is also evidence that it can be an effective way to strengthen health professionals' capacity to address nutrition and other health-related issues.[2,13,16-19] For example, one study found that an online training increased capacity of nutrition professionals to address childhood obesity in the U.S. by increasing participants' knowledge, skills, and self-efficacy.[18] Another study showed online distance education was an effective way to produce sustainable improvements in skills of health practitioners in Bangladesh.[19] We explored whether online training could build capacity of health professionals in IYCF worldwide.

Needs Assessment and Course Development

To determine whether online training would be a practical or acceptable solution for providing CPD to UNICEF professionals worldwide, a needs assessment was conducted of the target audience of approximately 300 nutrition staff who were located in 140 countries.[20] Anticipating possible technical barriers to access, respondents were asked to report the speed and reliability of their internet connectivity and to test their ability to access sample content (eg, text, videos, audio). Needs and goals related to their own capacity-building and desired content for professional development in IYCF were also assessed. The results indicated online training was both feasible and acceptable, and that this audience desired CPD in a wide variety of IYCF topics.[20]

To develop the training, IYCF experts at UNICEF, recognized for its expertise and global reach in child nutrition, collaborated with nutrition experts at Cornell University, who had an existing online CPD platform to deliver the training. Six IYCF and nutrition experts from both organizations, all of whom were PhDs in nutrition, MDs, and/or RDs, had extensive experience in content development and teaching and were able to transfer their knowledge and skills to an online environment. An online CPD course, “Programming for IYCF,” was then jointly developed by these experts to cover current IYCF concepts, and program development, implementation, and evaluation, for nutritionists and other health professionals.

Course Description and Reach

Results of the needs assessment[20]; an existing programming guide in IYCF based on evidence, lessons learned, and best practices[12]; and UNICEF's experience with in-person CPD on IYCF provided the rationale for the chosen topics. The course consisted of 14 units: 11 audio-visual lectures, each approximately 1 hour long, and three hypothetical case studies based on real programming scenarios to provide practice in applying course materials. The course was hosted and delivered on Cornell University's existing online CPD platform for nutrition and health professionals. Course objectives and topics are shown in Table 1.

TABLE 1.

Programming for Infant and Young Child Feeding (IYCF): Course Objectives, Topics, and Unit Titles

Course objectives

After completing the training, participants should have:

1. Awareness of up-to-date knowledge about IYCF

2. Ability to interpret data and describe population-based problems related to IYCF

3. Ability to recommend appropriate IYCF research, programs, and policies to supervisors, partners, and national governments

4. Awareness of best practices to coordinate/facilitate/manage action on recommended IYCF research, programs, and policies

5. Awareness of best practices to incorporate IYCF initiatives with emerging issues

Course topics and unit titles

Essentials

1. Essentials of malnutrition

2. Role of IYCF in child survival, growth, and development

3. Essentials of breastfeeding

4. Essentials of complementary feeding

5. Essentials of nutrition for women*

Programming

6. Comprehensive IYCF programming

7. Selected interventions for improving breastfeeding

8. Selected interventions for improving complementary feeding

9. Monitoring to strengthen IYCF programs*

Special topics

10. IYCF in emergencies

11. IYCF in the context of HIV/AIDS

Case studies

12. Case study—country 1

13. Case study—country 2

14. Case study—country 3

Added in 2016.

IYCF indicates infant and young child feeding.

The course was asynchronous and self-paced, so participants could register online at any time. Each person was given 4 months from their registration date to complete the course. Reminders were emailed to encourage course completion. To complete the course, participants had to score 100% on 14 unit tests and a final exam and were allowed to re-take them as many times as necessary. Once a participant had successfully completed all unit tests, the final exam, and a postcourse survey, a personalized certificate of completion worth 14 contact hours was automatically generated. The certificate was issued jointly by both UNICEF and Cornell University.

The first version of the course was launched in 2012 and offered for free to UNICEF nutrition staff via an email announcement. UNICEF staff were encouraged to invite others to take the course, especially their professional in-country counterparts including representatives from governments, other UN agencies, or non-governmental organizations (NGOs).

The stated goal for the course was to reach at least 200 of the potential 300 UNICEF global nutrition staff within the first year, which would result in a cost of about $350 per person to deliver the course, much less than the cost per person to deliver it in person. Instead, an astounding 5500 people from 154 countries registered. Within 6 years, over 14,000 people (including 1200 UNICEF staff) had registered and more than 4500 had completed the full course. The majority of course registrants (80%) and completers (88%) were from 115 different LMICs. This indicated that the course tapped into an unmet need for this type of CPD in LMICs and globally. This overwhelming interest was observed despite minimal direct promotion that primarily reached staff in UNICEF. Roughly half of course completers reported that they heard about it from colleagues; less than 20% mentioned listservs, websites, and/or conferences.

Course Evaluation and Results

To understand more about learner satisfaction, learning, and application, data were collected through three anonymous online surveys (see Supplemental Digital Contents, http://links.lww.com/JCEHP/A110, http://links.lww.com/JCEHP/A111, http://links.lww.com/JCEHP/A112, respectively). The course evaluation surveys were collaboratively developed by those who created the content for the online course and were familiar with learning assessment. Cornell University developed the initial survey questions and UNICEF provided input on how to adapt the questions for the target audience of their staff. The surveys, and the course, were in English, the most commonly used among UNICEF's official languages.

The three surveys differed in timing, target respondents, response rates, and purpose (Table 2). The postcourse survey assessed immediate course satisfaction and learning; the follow-up survey assessed whether and how course completers applied the knowledge and skills they gained from the course 9 months later; and the partial-completer survey assessed why some individuals who started the course did not complete it within the 4-month timeframe. Employment information was also collected in each of the surveys.

TABLE 2.

Survey Timing, Respondents, and Response Rates for Program Evaluation

Survey	Survey Timing	Target Respondents	Potential No. of Respondents	No. of Respondents	Response Rate (%)
Postcourse survey	Immediately after completing all course units	Individuals who completed all 14 course units	845	835	99
Follow-up survey	9 months after completing all course units	Individuals who completed all 14 course units	565	312	55
Partial-completer survey	4 months after registering for the course	Individuals who completed 6 or more units	246	72	29

Although we had postcourse surveys from almost all completers since the course was launched, the follow-up and partial-completer surveys were only implemented after the 2016 course revision. Thus, we chose to analyze surveys completed within the 1-year period after the course revision, once all three surveys were implemented. This facilitated our understanding of how the new course material and topics were received and applied by course completers.

Most survey questions provided Likert-scale type options and a few questions were open-ended. The questions related to satisfaction, perceived learning, and application of knowledge and skills were based on similar items with established face validity used in a different online course for nutrition professionals.[18] Questions were adapted to reflect the current course's objectives and content.

Responses from these surveys were compiled and analyzed. Employment information and Likert-scale questions were analyzed using Microsoft Excel and SPSS. Open-ended questions were analyzed using thematic analysis[21] and ATLAS.ti, a program designed for qualitative analysis. Cornell University’s Institutional Review Board deemed this project exempt from review.

The course evaluation gave insights from a wide variety of course participants (Table 3). The largest proportion of survey respondents worked for NGOs, governments, universities, and UNICEF. Among partial-completer survey respondents, a relatively larger proportion worked in hospital/clinical settings. More than 60% of post course and follow-up survey respondents worked in Africa, followed by over 10% in South Asia.

TABLE 3.

Employment and Geographic Location of Survey Respondents

Characteristic	Postcourse Survey Respondents (n = 835)	Follow-Up Survey Respondents (n = 312)	Partial-Completer Survey Respondents (n = 72)
Employment, n (%)*
Non-government organizations	395 (48.4)	131 (42.3)	24 (33.3)
Government	91 (11.2)	41 (13.2)	6 (8.3)
University/college	89 (10.9)	38 (12.3)	6 (8.3)
UNICEF	74 (9.1)	29 (9.4)	10 (13.8)
Hospital/clinical	70 (8.6)	10 (3.2)	15 (20.8)
Self-employed/consultant	25 (3.1)	21(6.8)	2 (2.8)
UN agency other than UNICEF	19 (2.3)	14 (4.5)	3 (4.2)
Business/private sector	16 (2.0)	15 (4.8)	4 (5.6)
Other	37 (4.5)	11 (3.5)	2 (2.8)
Geographic regions or countries, n (%)*†
West or Central Africa	273 (34.4)	96 (31.9)	16 (23.2)
Eastern or Southern Africa	209 (26.4)	90 (29.9)	11 (15.9)
South Asia	103 (13.0)	42 (14.0)	15 (21.7)
Middle East or North Africa	71 (9.0)	15 (5.0)	9 (13.0)
East Asia or the Pacific	18 (2.3)	7 (2.3)	4 (5.8)
Europe or Central Asia	16 (2.0)	10 (3.3)	3 (4.3)
Latin America or the Caribbean	11 (1.4)	4 (1.3)	2 (2.9)
High-income countries	44 (5.6)	16 (5.3)	4 (5.8)
Other	48 (6.1)	21 (7.0)	5 (7.2)

Percentages were calculated based on the number of responses for that particular question.

The first seven regions listed are official UNICEF regions. The region called “High-income countries” is not a UNICEF region but reflects the UNICEF national committee countries.

UNICEF indicates United Nations Children's Fund.

In the postcourse survey, respondents felt that the online learning modality was either “equally effective” (56%) or “more effective” (31%) than a face-to-face workshop. In addition, almost all respondents rated the course as “excellent” (55%) or “very good” (38%). Nearly all (99%) respondents reported that they learned either “a lot” (≥65%) or “some” from each topic (Fig. 1A). Furthermore, over 95% of postcourse survey respondents felt that what they learned was applicable to their work and they intended to apply the information, tools, and resources.

FIGURE 1.

A, Postcourse survey results (n = 835) for the question “how would you rate the amount you learned from each section of the course, using the scale provided?” B, Follow-up survey results (n = 312) for the question “how would you rate the amount you applied from each section of the course to your work, using the scale provided?” aThe three case studies were grouped together in the evaluation and are, thus, shown as a single topic rather than separately. IYCF indicates infant and young child feeding.

In the follow-up survey, 90% or more of respondents reported that they applied either “a lot” of or “some” information from eight of the 14 topics (Fig. 1B). More than 70% applied “a lot” of or “some” information from the remaining topics. Respondents also reported using knowledge gained from the course in a variety of ways, including in presentations or reports; to interpret problems related to IYCF; and to make recommendations for research, programs, or policies. The vast majority of respondents (91%) shared course information with colleagues and agreed (84%) that having a personalized certificate of completion was useful to them.

In the postcourse survey, responses to open-ended questions fell under three broad categories (Table 4). Increased knowledge and understanding of IYCF resulted in better skills and capacity to carry out interventions and implement concepts learned. The course was regarded as high-quality and useful; more similar courses were requested. Respondents planned to teach community members or train colleagues, to recommend the course to others, and to improve programming or practices.

TABLE 4.

Categorized Responses From Open-Ended Questions on Postcourse and Follow-Up Surveys

Response Categories (in Italics) and Subcategories	No. of responses*	Example Quotations from Each Category
Postcourse survey responses
Improved knowledge and skills		I am better informed in facing challenges and answering questions that come up during work.I have learned a lot about IYCF, most of the information in this course we were not  taught in college. I am now able to confidently take care of patients especially  women and children.
Knowledge or perspective	173
Capacity or skills	58
Important existing course topics	67
Filled gaps, refresher	17
Course was useful/good		This course was refreshing and was like a complete checklist of knowledge and a very interesting learning experience.Send me more online course works and together we shall improve on lives of  children.
Good course, empowering	73
Useful, practical, or helpful	60
Grateful, thankful	53
Interesting, enjoyed course	29
Wants more courses	26
Recommends to others	20
Application of course to work		We have a lot of malnutrition cases in my country, so I am planning to educate the community how to manage this condition.This course will help me broaden my IYCF interventions in communities where I  work.
Impact on job/work	67
Applying, teaching, or counseling in community	57
Implementation, programming, and policy	43
Improve practice	21
Career advancement	13
Follow-up survey responses
Impact of course		I have been able to do counseling to many HIV women feeding their children. I show them the importance of breastfeeding their children for 6 months.I conducted a refresher program for my colleagues by including the contents of  the course, which they found extremely useful. All of my colleagues have  now completed the course.
Improved skills and knowledge	87
Important existing course topics	30
Improved practices or care	27
Supporting communities and programs	21
Who has been impacted	16
Training others	9
Course was useful/good		Very essential and useful course. I will recommend every humanitarian worker, most especially those in nutrition, to take the course.
Useful, helpful	74
Grateful, thank you	32
Excellent, fantastic, important, etc.	25
Recommend course to others	11
Helped my work or career		With the completion of the course and certification I am now see[n] as a well knowledgeable person in nutrition in the society and work place.
Competence, expert	18
Confidence	6
Promotion, advanced career	6
Wants more…		[The course] has enhanced my knowledge… More advance[d] course with better certificate and credit hours would excel my professional life. Request you[r] team to design such [a] course for professionals like me.
Courses and training	27
Diploma, advanced degree	9
Languages offered	4

Responses were optional, and people could respond more than once.

IYCF indicates infant and young child feeding.

In the follow-up survey, responses fell under four broad categories (Table 4). Respondents reported that applying their skills and knowledge helped them improve practices and outcomes, teach community members directly, train other professionals, and change program activities. The course was highly regarded, with representative words such as “essential,” “fantastic,” and “important” used to describe it; many had recommended the course to others. Respondents also felt that the course had helped their work or career by increasing their competence and confidence in IYCF, and even led to promotions in some instances. Changes requested included adding more topics in this course, developing additional online courses, and in a few instances, offering it in more languages (Table 4).

In the partial-completer survey, “not having enough time” was stated as the reason for not completing the course within 4 months by 74% of the respondents. Other reasons respondents gave were that certain units were not relevant to their jobs or did not interest them. Illness, work, and family commitments, and poor internet connection were also mentioned. Nevertheless, all of the partial-completer respondents reported that the course was excellent (39%), very good (46%), or good (15%). Ninety-four percent were still interested in taking units they had not yet completed.

Implications for Online CPD

Since its launch, the course has averaged over 2000 registrations per year, exceeding the original goals and expectations. This is evidence of the critical need for CPD to build capacity of health professionals in nutrition, particularly in LMICs.[1,15]

The course has had a high completion rate for a free online course, with a consistent rate over time of about 32%. In comparison, reports from 236 other massive open online courses, where thousands of people have enrolled, showed a median 7.7% completion rate over 4 years.[22] Our high completion rate and favorable survey responses are testaments to the quality and value of this course, and to the need for more online education for professionals globally.

This course has had a reach far beyond UNICEF staff alone; many health professionals, employed by NGOs, governments, universities, and clinical settings have also completed the course. There were some differences in the proportions who completed the course. Those employed by UNICEF, NGOs, and governments, who worked primarily in community settings, more often completed the course, possibly because they were more likely to apply the course information to a public health nutrition context. Those in clinical settings, who typically provide care for individual clients rather than community populations, were less likely to complete the course, possibly because some community-based programming topics were less relevant to their work. Conversely, more clinicians completed the partial-completer survey in which most respondents reported insufficient time to complete the course, indicating they may have simply had less time available to take an online training.

The online delivery method was found to be an effective and well-received way to deliver IYCF training to nutritionists and other health professionals in LMICs. It was also relatively low cost, especially compared to the $2000 per person estimate for in-person workshops. Although the cost per person varied slightly over time as more people completed the online course, the cost was approximately $75 per person at the time of analysis and had been at this level for several years. This indicates online training can provide an affordable and acceptable alternative for organizations with limited training budgets. Combining online with in-person training has been recommended as a cost-effective approach for continuing education for professionals.[1,19]

Completers reported improved knowledge immediately after completing the course and application of the content 9 months later (Figs. 1A, B). One limitation in generalizing the degree of application is that non-responders might not have applied the content to the same degree as respondents. Data were also self-reported and there was no mechanism in place to verify the information provided by the respondents. Although anonymous, there may have been a bias toward reporting a higher degree of application among the 9 to 14% who worked for UNICEF to please the course organizers, who provide global leadership and guidance for these staff without being their direct supervisors.

Course satisfaction, learning, and application among noncompleters are also unknown. Those who did not partially or fully complete the course within the evaluation time period were not surveyed because the aim of this program evaluation was to determine what completers learned and applied. Possible reasons for noncompletion include that the content did not match their needs or contexts, the content level was too basic or too advanced, and/or there were technical problems accessing it. Even with relevant content, if employers did not require it, there may have been no obligation to complete the entire course. Some non-completers may have only had interest in a few units, which is supported by the fact that some units had a higher number of completers than others. For example, the topics related to the essentials of IYCF were more often completed than those related to programming in IYCF (data not shown). It is likely that the topics that focused on the essentials of IYCF were relevant to all health professionals, whereas the programming topics were more relevant to UNICEF staff who were closely involved in IYCF program management.

Noncompleters may have also never intended to complete the course. For example, trainers in university or other settings may have enrolled in the course to determine whether the course content may be useful to assign to students and never intended to complete any units themselves. The fact that the course was free could also have affected the level of engagement and completion rates. Research on massive open online courses suggests that many people enroll but never intend to complete the entire course; however, this should not be interpreted to suggest that these courses are not meeting learners' needs.[23]

In general, there is a need for more information about why some professionals, especially in LMICs, register for online CPD, but do not engage to a significant degree with the course. Obtaining such information through a separate evaluation would help in the design and implementation of future CPD for this audience. Based on our findings and those of others,[1,19,24] offering a hybrid course that combines online with occasional in-person training or mentoring may increase overall engagement in CPD. Our program evaluation sets the stage for future studies on how to improve learner engagement and how increasing the capacity of health professionals in LMICs via online training affects their practice and ultimately health and nutrition outcomes in those countries.

Lessons Learned

Overall, the online course improved knowledge and application of basic IYCF concepts in nutrition programming for health professionals working in multiple settings and organizations. Our program evaluation provided not only evidence of program effectiveness, but also ways to improve course delivery and content, some of which have already been implemented. For example, after seeing how many non-UNICEF health professionals were enrolling, we realized there may not be a need for all registrants to complete all units. Thus, changes were made to award certificates of completion for individual units, because, based on survey results, the certificate of completion was important to learners. The 4-month deadline to complete the course was also eliminated to allow all registrants, not just course completers, unlimited access to the content, including any updated information.

One benefit of an online course is that the content is all updated in one place and can become immediately available for all course registrants. We reviewed and updated the entire course content in 2016, the year before the surveys were administered, allowing us to reflect the latest evidence at that time. Online platforms can deliver any content, and over time other health-related topics of high interest, such as non-communicable diseases, guidelines on physical activity, or health issues related to pandemics could be delivered on the same platform. Conversely, content is not platform-specific and can be transferred to other online platforms. In fact, the IYCF course has already been successfully moved to another platform and is now being delivered on UNICEF's online professional development platform at http://agora.unicef.org. Therefore, the efforts that go into developing online content can be leveraged to serve as a model for developing similar content on the same platform and/or for converting the content to other platforms. Finally, the context of pandemics such as COVID-19 indicate that the use of online platforms to build capacity of professionals may become even more important in the future, and increased investment in online training by program planners will be required to meet this need.

CONCLUSION

In summary, there is an unmet need for CPD among health professionals in LMICs. Interest in and evaluations from our online course demonstrate that such trainings are desirable and can at least partially meet this need in an accessible, affordable, and acceptable way. Increased use of such trainings has the potential to reach a large number of health professionals worldwide and build their capacity in a wide variety of health- and nutrition-related topics. This in turn will contribute toward improving the quality of practices and care at the local level and ultimately, the health and nutrition of vulnerable populations.

Lessons for Practice

■ There is an unmet need for continuing professional development (CPD) among health professionals in low- and middle-income countries (LMICs).

■ Online training is a practical and acceptable yet underused solution for providing CPD to health professionals in LMICs.

■ Nutrition and other health professionals who completed an online training in infant and young child feeding (IYCF) reported increased knowledge, skills, and application of IYCF practices, indicating that online CPD has the potential to build capacity and improve health-related practices in LMICs.