A process evaluation of the Communication for Healthy Communities adolescent health program in Uganda.

This study is a process evaluation of an adolescent-focused intervention of the USAID Communication for Healthy Communities program, in Uganda. We used mixed methods including observation, consultations and review of program documents to collect data on program coverage, reach and factors influencing implementation. Findings show that program activities were successfully implemented through collaborative partnerships with service partners and the community. Interpersonal communication complemented by mass-media messaging was effective in reaching and empowering adolescents with health information to make informed choices for behavior change. The program used theoretical frameworks to guide targeted interventions through audience segmentation and community empowerment. Targeted mass-media messaging and placement was found to be pertinent for program reach. Working through existing community structures is important for an effective reach of health promotion programs. Lessons identified for scaling-up adolescent health programs include the need to harmonize training and deployment of community champions by development partners, recruit audience-specific influential champions and link income-generating activities to health education interventions. There is thus need to collaboratively develop and institutionalize effective monitoring and evaluation strategies during program inception and design phases for appropriate accountability, ownership and a continuation of gains. � The Author(s) 2019. Published by Oxford University Press.

Introduction

With the increasing population of adolescents in sub-Saharan Africa, the design of targeted adolescent sexual and reproductive health programs has recently become a priority for governments and international development partners. This follows the realization that adolescents contribute to the alarming health indicators of increasing HIV prevalence and teenage pregnancy globally [1, 2]. While a significant proportion of Uganda’s population are adolescents, adolescent sexual and reproductive health services are limited and do not address their needs [3, 4]. This is partly due to the limited understanding of the sexual and reproductive health needs of adolescents as distinct from often generalized youth [5]. In Uganda, in 2016, adolescents had a 25% teenage pregnancy rate, and a 1.8% HIV prevalence for girls and 0.4% for boys [6]. Adolescents continue to face unmet sexual and reproductive health needs (e.g. 30% contraception use in Uganda in 2016) [6] and maternal and child mortality associated with unsafe deliveries [7, 8]. These challenges are often aggravated by early marriages and poverty that force adolescents to engage in multiple transactional sexual relationships with older partners for material gain and the belief that relationships with these partners are stable, however, this often leads to gender power imbalance and affects adolescents’ ability to negotiate safe sex. Additionally, adolescents of similar-age sexual relationships face risks of engaging in unprotected sex which often leads to HIV, unplanned pregnancies and teen motherhood. This creates more risks of pregnant girls being chased from home, thus increasing the potential for engaging in transactional sex and early marriages, they often have limited financial stability to care for children and child headed households, and the pregnant girls also drop out of school hence minimizing the opportunity to reach their full potential [9, 10]. Adolescents’ beliefs regarding sex, contraceptive choices and reproductive health are largely based on information from peers and often characterized by inadequate knowledge, myths and misconceptions [11, 12]. However, many adolescent health-targeted programs are not evaluated for their contribution toward addressing adolescents’ needs. This study evaluated the ‘Accelerating the Rise in Contraceptive Prevalence’ (ARC) adolescent health interventions of the United States Agency for International Development (USAID)-funded, Communication for Healthy Communities (CHC) program in Uganda.

The CHC program was developed to support the Government of Uganda and its implementing partners to address adolescent sexual and reproductive health needs. CHC is a 5-year program implemented by a cooperative agreement signed in June 2013, between USAID Uganda and FHI 360 (Family Health International). The program aimed to design and implement quality health-communication interventions to contribute to a reduction in HIV infections, total fertility, maternal and child mortality, malnutrition, malaria and tuberculosis [13]. A CHC-implemented process evaluation of adolescent sexual and reproductive health interventions guided program scale-up. A process evaluation provides information on program implementation, which is important for interpreting program outcomes and informing future efforts including generating critical lessons for impact evaluations [14-16]. This study presents the process-evaluation findings on how the CHC program’s prioritized health (and recommended) behavior-change actions were received and adopted by adolescents.

The overarching CHC program implements an integrated campaign called ‘Obulamu? [How’s Life?]’. The campaign platform uses a Life Cycle approach to integrate health communication messaging across age groups and public health topics. The program incorporates interventions that address the norms of gender inequity, which act as barriers to health services uptake [13]. The integrated Obulamu? campaign delivers holistic messages relevant to a person’s stage of life rather than focusing on a fragmented disease approach. The four life stages addressed in the campaign include: (i) young adults (18–30 years) in relationships; (ii) pregnant women and their male partners; (iii) families with children (0–14 years) and targets caregivers of under-five children and (iv) adolescents (15–19 years) [13] as indicated in Fig. 1.

Fig. 1.

Obulamu? Life Cycle approach.

Through its campaign platform, CHC implements as a sub-program the adolescent-focused component of ARC program in Uganda, which addresses adolescent contraception needs [17]. This process evaluation focused on ARC’s adolescent sexual and reproductive health interventions to understand their current effect for future programming. The use of process evaluation to understand ARC’s contribution was essential to avoid a ‘black-box evaluation’ (measuring and attributing outcomes to a program without exploring how the program was delivered) [18, 19].

Materials and methods

Program description

The ARC program, specifically implemented in 2016–17, primarily targets adolescent girls aged 15–19 (in and out of school), and young women aged 20–24 and their partners in five districts (Kamuli, Iganga, Mayuge, Namutumba and Luuka) of East Central Uganda. These districts have the highest rates of teenage pregnancy [11]. The secondary target audiences include adolescent boys aged 15–19 years and young men aged 20–24 years, parents and guardians of adolescents, health workers and religious leaders. Using the 360-degree communication approach indicated in Fig. 2, the campaign used a multi-channel approach employing a variety of high-dose and high-intensity interpersonal communication activities including community dialogs [13]. Interpersonal communication activities were complemented by radio, TV/video, outdoor media and social media placements [13].

Fig. 2.

CHC's 360-degree communication approach for the ARC program.

The design and implementation of the ARC program were grounded in the underlying ideologies of the Socio Ecological Model illustrated in Fig. 3. The model looks beyond individuals to their social context including interpersonal, institutional, community and policy environments [13, 20, 21]. The social ecological paradigm is rooted in core principles concerning the interrelations among environmental conditions, human behavior and well-being. Environmental settings are characterized as having multiple physical, social and cultural dimensions that can influence a variety of health outcomes [22]. The Social Ecological Model was used to analyze the context in which the target audiences made decisions, accessed services and related socially to address barriers to change using theory- and evidence-based programming [13].

Fig. 3.

An illustration of the social ecological model. Source: Mckee et al. [20].

Study design

This study used a process-evaluation design. We used a mixed-methods approach combining quantitative and qualitative data collection methods as recommended for complex interventions [23].

Data collection methods

Data collection included a review of CHC program records, direct observation during site-monitoring activities and consultations with program staff.

Study team

Planning a process evaluation requires sufficient expertise and experience to determine and achieve the aims [23]. This study drew expertise from a range of relevant disciplines including public health, behavioral science, health promotion, primary care, sociology, monitoring, evaluation and research. The experts provided practical experience in the design and implementation of adolescent sexual and reproductive health programs from various settings, and objective judgments in the process evaluation.

Review of program records

The review of ARC program records (availed on third-party request) entailed an assessment of the program’s theoretical approaches, and documents related to activity implementation and research (see below).

Assessment of program theoretical approaches

All public health interventions reflect explicit and implicit theories on how a course of action will solve a perceived problem [24, 25]. Theories should clearly specify intervening processes or mechanisms that link the program’s activities and outcomes [19, 26]. For this study, we were guided by the process-evaluation model in structuring and presenting the findings. However, using the ARC program documents, we assessed the theoretical models and constructs embedded in program design and implementation. This included identifying the explicit and implicit theoretical models of the ARC program on the premise that no single theory explains all, but a mix of theories is often required to understand factors such as the different types of audiences, and contextual considerations at every stage/phase of a program.

Assessment of documents for program activity implementation

Process evaluation encompasses an assessment of the following: (i) program implementation, (ii) specific intervention activities, (iii) context surrounding the activities, (iv) responsible personnel, (v) target audiences and (vi) level of effort [27, 28]. To understand context, we first reviewed the CHC and ARC programs’ inception documents including the technical proposal and health communication audit report. Next, we reviewed documents related to activity implementation, individual activity reports, monthly and quarterly reports, PowerPoint presentations and minutes from planning meetings.

Assessment of documents for program research

We reviewed ARC research documents to understand the program’s diffusion in the community and community awareness. The reviewed documents included formative assessments, evaluative surveys (quantitative/cross-sectional), qualitative rapid assessments and participatory action research reports. National surveys, including Uganda Demographic Health surveys, were reviewed to corroborate the ARC program research reports. The aim was to understand the program’s progress in addressing performance measures/indicators.

Direct observation during on-site monitoring

The ARC program staff shared their field observation experiences and program reports that highlighted observation findings were used to draw insights for this study. Approximately 12 process-evaluation observations were conducted using a checklist during on-site monitoring visits to high-volume health facilities. This is because the overall ARC program design was refocused on high-volume sites such as health centers three and four, thus the program obligations were to implement and monitor activities in such sites.

Consultations with ARC program staff

The staff who participated in ARC program implementation were identified and purposively selected to participate in this study. A total of 10 program staff (five male and five female) agreed to participate in the study consultations. The staff verbally agreed and voluntarily consented to participate in this study after a thorough explanation of the study purpose. Consultations with ARC program staff provided a clear understanding of the program’s implementation approaches.

Evaluation outcomes and data analysis

The evaluation outcomes of interest were to understand ARC’s program design and coverage factors that influenced program implementation, and the program’s effectiveness in reaching the intended target populations. To ensure a structured data analysis based on the themes and code categories identified in program documents, we adopted Linnan and Steckler’s (2002) process-evaluation components. The components include a description of the program context, reach, dose delivered, dose received and fidelity [18]. We analyzed data from January 2014–December 2017, including the overall CHC program’s inception and design periods, and implementation of the ARC sub-program. From the outset, adolescents were identified as a key target audience in the overall CHC program design, thus the focus on the retrospective period of 2014–17 aimed at providing a comparative edge of describing program effects to the target audience with consideration of adolescent interventions implemented before, during and after the ARC program. The ARC program initiated in 2016 offered targeted programming in a select geographical area of the country marked with high teen pregnancy and teen motherhood.

Qualitative thematic content analysis of the available program documents was conducted [29, 30]. Thematic content analysis refers to a systematic review of written documents produced by ARC program staff and other stakeholders including village health teams, peer champions, community leaders and community members during program design and implementation [30]. From the review of written documents, themes, code categories, words and phrases that captured salient program elements were developed into a common analysis framework reflected in the study evaluation questions (Table I). Quantitative data from cross-sectional surveys and rapid assessments that addressed the study objectives were analyzed using the program’s Monitoring, Evaluation and Learning Plan including the program’s performance indicators on contraception use [31]. The analysis focused on performance indicators such as exposure to family-planning messages, knowledge, awareness and contraception prevalence rates among adolescents. The program planning/implementation team and community members (especially the target population–adolescents) were the main units of analysis. The use of both qualitative and quantitative data analysis techniques allowed for data saturation while informing the process evaluation.

Table I.

Process-evaluation questions for CHC’s ARC program

Components	Questions	Indicators	Method
Context	What were the policy, social and economic environmental contexts before and during program implementation? What changes were made in the policy environment for adolescent health? Were changes informed by evidence? What ARC health-communication programs or messages were adolescents exposed to before and after the interventions? How were various stakeholders involved in the ARC program design and implementation?	Existing adolescent health policies addressing adolescent SRH needs Adolescent access to SRH services Adolescent exposure to health messages Level of stakeholder engagement and skill development for empowerment	Assessment of theoretical approaches Assessment of design and practical activity implementation documents Consultations with implementation team
Reach	To what extent is the program reaching the adolescents? Were sub-groups within the adolescent population being missed? Were some sub-groups reached more than others? Were unintended groups taking part and benefiting from the ARC program? Were changes made during program implementation? To what extent were the changes informed by evidence to ensure effective reach?	Percentage of intended participants, disaggregated by sub-group Percentage of unintended users Number of changes made during program implementation to ensure effective reach	Assessment of practical activity implementation documents Observation of activities during on-site monitoring Consultations with implementation team
Dose delivered	How much of the program was delivered? What was delivered inconsistently or omitted? Why? To what extent were the planned intended message dose and intensity delivered? Did any factors affect the delivered dose?	Percentage of the target audience who reported having seen or heard messages	Assessment of practical activity implementation documents Media monitoring reports
Dose received	What was the average dose received by adolescents? What parts of the intervention were not received consistently? Why? What was the difference in results (based on the evaluation) for intervention segments delivered consistently and inconsistently?	Percentage of adolescents who demonstrated comprehensive knowledge of contraception, approved of desired behaviors and/or health services on contraception, and intended to adopt the same. Percentage of sexually active adolescent girls aged 15–19 who used modern contraception methods	Assessment of program research documents
Fidelity	How were the program interventions and messages linked to theoretical methods in practice? Was the program implemented with fidelity and quality? Were the program interventions implemented as planned? Were there program interventions that did not work? How did the program use evidence and feedback on what did not work to learn and adapt the activity implementation?	The degree to which the message was linked to theoretical methods and practical applications, and determinants	Narrative analysis and assessment of feedback documents Assessment of research documents

Data analysis software (NVivo for qualitative data and SPSS for quantitative data) was used for data structuring. We used NVivo 11 to facilitate the data coding process of audio-recorded and verbatim transcribed qualitative data drawn from the program documents and reports. Thematic analysis approach was used to organize qualitative data into themes and sub-themes used to develop emerging concepts and interpret meaning of the data. This approach was chosen to help in gaining direct information from the study participants without imposing preconceived ideas. Analysis codes were developed by first picking out potential themes identified from the question guides, and later the identified potential themes were refined or regrouped by investigators in collaboration with the coding assistants during reading and re-reading of emerging data. Analysis codes such as program reach, targeted mass-media placements, training and deployment of champions, and income-generating activities and health education were developed to guide generation of emerging concepts that supported the interpretation of data for this study. For quantitative analysis, we reanalyzed data from the program survey datasets using SPSS 16 to confirm the performance indicator data used in this study. Sample weights were calculated based on the probabilities of parish, village, household and household member being selected. The sample weights were used to determine the percentages drawn from the ARC program reports. Descriptive statistics and frequencies disaggregated by gender and age were run based on the program performance indicators that focused on the dose delivered and received by the target audience such as those who were exposed to family-planning messages, those who had knowledge of the use of modern contraception methods, those who approved the use of contraceptives and contraceptive prevalence rate. Only data that focused on adolescents aged 15–19 was included in this study.

Process-evaluation questions for CHC’s ARC program implementation

In contextualizing the process-evaluation components, this study explored the evaluation questions highlighted in Table I.

Ethical considerations

Ethical approval for this study was sought as part of the overarching CHC program process implementation and evaluation which included components of field observation, therefore covering all study participants (ARC program staff and adolescents). The FHI 360 institutional review board (the USA federally registered Protection of Human Subjects Committee) waived written consent under the study reference number 616862-1. The Ugandan Government-accredited Makerere School of Public Health Research Ethics Committee also waived written consent under reference 259. A waiver of written consent was sought to minimize the risk of a loss of anonymity because signed informed consents would have been a link to the participants’ identity in this study. Additionally, participant confidentiality was assured by ensuring that all program reports and quotes do not contain any identifiers such as names of the respondents. The data collection staff also underwent appropriate training on the objectives, questions, community entry and ethical procedures for participant recruitment, confidentiality and data handling and storage. The participants were told the purpose of the study and given the opportunity to ask questions before agreeing to participate. Verbal informed consent was requested from ARC program staff and adolescents before being interviewed. The date and time of verbal consent were noted for each participant accompanied by the responsible study staff member’s signature.

Results

Our evaluation assessed the practical experience of the ARC program’s design and implementation. The analysis focused on three main aspects: (i) context of program design, (ii) areas of program success and (iii) areas for strengthening the scale-up of adolescent health programs. Evaluation results should be interpreted with caution because they do not reflect the full impact of the overarching Obulamu? campaign and the whole CHC program on adolescents. Rather, the central focus was on aspects that culminated in the ARC program’s design and implementation.

The context of program design

The program-planning documents indicated that the ARC program was developed following the alarming health outcomes among adolescents in Uganda. Uganda’s unmet need for family planning was among the highest in sub-Saharan Africa—31% among adolescent girls aged 15–19 [32, 33]. Exacerbating the challenge of unmet need was gender inequality affecting health outcomes by limiting women’s decision-making power within families and their ability to access health services [32]. The overall effectiveness of health communication interventions in Uganda, even in late 2012, was identified as unclear and not well translated into health outcomes [32, 34]. The fragmentation of health communication efforts was identified as a major problem. This was linked to the lack of collaboration and coordination among diverse government and development partners responsible for designing and delivering health communication. Although health communication campaigns were found to be targeted to specific audiences, they had limited coverage and did not reach beyond urban centers. Messaging focused on top-down messages delivered through limited channels relying heavily on printed materials, although only slightly more than a quarter of Uganda’s population is literate [32, 33]. Mass-media campaigns were not adequately intensive or linked with interpersonal communication and other community-based approaches in mutually reinforcing ways [32]. Thus, local communities were not engaged as partners in their own behavior change, which made it difficult to address gender and other social norms that negatively affect health behaviors and service demands. Within facilities, health workers’ limited interpersonal communication skills were found to contribute to a perception of unfriendly services in many communities. Existing national health communication strategies before the project intervention were still in draft form, and some were outdated [34]. Existing health communication materials and activities conveying health-related social and behavior change communication (SBCC) messages were occasionally contradictory. While strategies covered gender issues, actual materials negligibly included considerations for social systems and relationships that reinforced existing behaviors. Capacity gaps were identified as affecting the operationalization of working groups and task forces, for example the adolescent sexual and reproductive health working group. Services targeting adolescents were limited and/or poorly resourced with staff and commodities in general [3, 34]. These contextual factors (summarized in Fig. 4) and the realization that there was a need to target adolescent sexual and reproductive health needs—specifically addressing communication on contraception use—led to the ARC program design.

Fig. 4.

A summary of contextual issues identified before and during implementation of the ARC program for adolescent and young adult females. IPC, interpersonal communication; HC, health communication.

Areas of program success

Program coverage

The findings showed that the ARC program was designed to increase knowledge and uptake of modern contraceptives among adolescents and young women of reproductive age in selected districts of Uganda. It aimed to increase adolescent and young adult females’ comprehensive knowledge of contraception and its benefits and equip them with information to make informed choices [11]. Embedded within the program was the explicit use of theoretical frameworks, including the Social Ecological Model, to define the primary target population and their key influencers. The program worked collaboratively with the Government of Uganda through the Ministry of Health and other development/implementing partners to conduct health communication interventions including mass-media messaging and interpersonal communication. For interpersonal communication, ARC planned to implement 150 youth bashes or edutainment events and orient and deploy 150 community champions. The mass-media plan was to develop and disseminate print and outdoor health communication materials such as posters, flyers, and billboards [35] and conduct monthly broadcasts on radio including, 180 radio spots, 180 DJ mentions and 90 DJ-led discussions called “Obulamu? Moments” on local radio stations with coverage in the five targeted districts.

Program reach

ARC had intended quantitative outcomes focused on the number of the target populations reached by interventions and qualitative outcomes focused on feedback from target audiences. Findings indicated that ARC staff conducted quarterly coordination meetings with service partners and developed joint implementation schedules linking demand generation and service delivery interventions. Working in this partnership, the ARC program trained/oriented and deployed 1303 champions (village health teams, peer champions and community leaders) who reached 202 060 adolescent girls and young women through interpersonal communication activities such as home visits, one-on-one sessions and small group discussions. Periodically, the program conducted quarterly progress review meetings with the champions where refresher orientation sessions lasting 5 days were conducted to equip the community champions with skills and information to communicate contraception use messages to the adolescents. Some adolescent girls and young adult women may have been reached more than once, and females from non-participating districts may also have been reached. For motivation during the quarterly progress review meetings, the community champions were given an allowance of transport refund based on the local public rate, a safari day or per diem allowance based on the government approved rates to cover their incidentals. More so, although religious leaders were identified at program start-up phase as potential key influencers of adolescents, we did not find clear documented evidence in the ARC program reports on adolescents’ engagement with religious leaders.

The ARC program conducted 53 community edutainment events that were locally known as Kadankes, wherein 8236 adolescent girls were reached with messages on contraception use. The program had 6240 radio placements on four local radio stations (Baba FM, NBS FM, Kamuli Broadcasting Service and Apex FM), and reached an estimated 400 000 adolescents every quarter [36]. Prioritized health messages included: using contraception to prevent teenage pregnancy, testing for HIV and receiving results, building healthy relationships with the opposite sex, delaying sexual debut, using condoms and seeking accurate health information.

ARC also reached secondary and unintended groups. For example, the community-based interventions and the radio broadcasts reached men and women beyond the targeted age group [36]. Implicit in program implementation was the use of other theoretical frameworks to reach targeted audiences. Using the Diffusion of Innovations theory, the program adopted targeted implementation of community interventions including materials distributed in hot-spots such as bars, bore-holes, markets and trading centers to reach laggards. The program used the Empowerment theory to build community capacity through collaborative partnerships and shared decision making. Interpersonal communication complemented by mass media was found to be an effective health communication approach by providing information that empowered adolescents to make informed health choices [37]. This is illustrated in the quotes below that were drawn from the ARC program documents showing qualitative assessments and implementation research findings.

I used to think that other family planning methods are for women who have given birth and want to space their children. But the peers helped me understand that family planning can be used by any female above 15 years and their information was confirmed by the nurse who gave me the injection. (Female adolescent, Namutumba district)

We get enough time with the champion so the level of discussion and understanding is far much better. Even the puzzling questions that someone would fear to ask in public, he/she will be free to ask since it is only two people. (Female adolescent, Iganga district)

The dose delivered and received

ARC program’s cross-sectional baseline and end-line surveys showed several self-reported and health service data changes among adolescents for family-planning services uptake. ARC evaluative surveys indicated that exposure to family-planning messages was 80% among adolescent girls. These females showed a slight increase in knowledge on the use of modern contraception methods (23% at baseline to 37% at end-line) (Table II). The proportion of adolescent girls who reported having one child decreased from 33% to 28% and those who reported having two or more children decreased from 10% to 6% [38].

Table II.

ARC program reach by performance indicators

Program performance indicators		ARC program evaluative assessments		CHC listening/ omnibus survey June 2017	CHC evaluative surveys
		Baseline percentage (%) October 2015	End-line percentage (%) September 2016	East Central region percentages (%)	Baseline percentage (%) 2015	End-line percentage (%) 2017
Exposure (heard or seen) to family-planning messages (pregnancy delay or birth spacing)		—	80	74	59	60
Knowledge of the use of modern contraception methods		23	37	—	38	56
Approve the use of contraceptives		55	74	74	85	82
Intention to seek contraceptive services in the next 6 months	Not married	40	58	86	—	—
	Married	57	66		—	—
Intention to use contraceptives in the next 6 months	Not married	39	53	47	33	42
	Married	54	67		35	44
Contraceptive prevalence rate (sexually active girls currently using any modern method of family planning)		—	—	33	17	25

ARC, Accelerating the Rise in Contraceptive Prevalence; CHC, Communication for Healthy Communities.

Although the CHC omnibus survey (2017) indicated that approval of the use of modern contraceptives was 74%, the contraceptive prevalence rate was relatively low at 33% (Table II) [39].

CHC evaluative surveys (2016) indicated that although there was a slight increase in prevalence at end-line, approval for contraception use remained high (baseline 85% and end-line 82%). There was a slight increase in the use of contraceptives (baseline 17%) and end-line 25%) and knowledge about contraception (baseline 38% and end-line 56%) (Table II).

Fidelity

ARC records showed that implementation was done as planned and changes effected using the collaborative learning and adaptation approach to address emerging adolescent needs. The Social Ecological Model was used as a theoretical framework in program implementation to segment target audiences. ARC instituted measures to continuously assess progress in the implementation of activities with quality and fidelity. The measures included continuous quality-improvement monitoring visits, progress review meetings, rapid assessments and client exit interviews during community edutainment events. The results from the program assessments indicate that trained peer champions were able to execute their interpersonal communication activities to influence adolescents in adopting health-seeking behaviors and increasing access to contraceptive services [40]. However, some champions revealed their difficulties in discussing some family-planning topics related to an age-group mismatch/barrier between the champion and adolescent. Monitoring reports indicated that 90% of the champions required additional support, for example refresher training, supervision and identification material access to effectively execute their work [41]. However, disaggregation of data in most program reports did not include targeted sub-group reach. This made it difficult to effectively evaluate and describe the ARC program’s reach.

Areas that did not work for the ARC program and where subsequent collaborative learning and adaptation occurred

Records indicated that the ARC program did not work in some areas and so staff used evidence and feedback to adapt and make changes in activity implementation. The program first targeted adolescent girls and young adult women without involving parents. However, most parents did not allow their adolescent girls to attend health edutainment events citing that they were avenues for girls to meet with boys. This affected the number of adolescent girls attending health gatherings. The program used this evidence to design a parent communication campaign. Community interventions were halted in January and February 2016 during Parliamentary and Presidential elections. The elections disrupted community health-edutainment events since they were mistaken as political rallies, and program campaign materials were vandalized. This disruption cost time in effective implementation of planned ARC activities. At inception, the program targeted all community champions, specifically village health teams attached to health facilities. This did not work because champion follow-up was not manageable and their work structure varied by implementing partners, thereby making champion reporting difficult. To improve follow-up, CHC changed the approach to focus on 10 champions per high-volume site and opted to work with existing community district structures to improve reporting.

Areas for strengthening in adolescent health programming

The study findings highlighted lessons for strengthening and improving adolescent health programming in Uganda.

Targeted radio placements for adolescents

Media channel mix was important for reaching adolescents. The key lesson from using mass media (especially radio and TV) was that identifying the time when adolescents listen to radios was important. Adolescents reported specific times and radio programs of interest on selected days that motivate them to listen. Effective channels of communication for reaching this target group were radio complemented by interpersonal communication [37]. Identifying high-volume listener times of a target audience is evidence that informed program adaptation to place health communication messages in youth-focused programs on radio and TV [36]. This is illustrated in the quotes drawn from ARC program documents showing qualitative assessment and implementation research findings.

Radios are more effective because those who do not read, can listen to radios and understand. (Female adolescent, Kyenjojo district)

The method which is not effective is the TV because we don’t own any, but the radio is very effective. (Male adolescent, Kyenjojo district)

Because on the radio the information is not in-depth, even posters always highlight the briefs on major issues compared to the village health teams who give detailed information. (Male adolescent, Lira district)

Harmonized training and deployment of champions by development partners

Monitoring reports indicated that although there was an appreciation of ARC’s interpersonal communication strategy to effectively engage community champions, efforts to train and deploy the champions were fragmented among development partners [41]. Harmonization of interpersonal communication interventions among implementing partners is critical for improving adolescent health programming. Harmonization of partner-led community champion efforts should be spearheaded by the Government. This is illustrated by the quote below as drawn from the ARC program documents showing qualitative assessment and implementation research findings.

You know the way the village health teams [interpersonal communication] strategy has been implemented, it has not been straight-forward because each partner who comes, they come with their own programs and they will all set targets for their programs, so we don’t have specific targets for the village health teams. Their work depends on the partner who comes and wants to use them. However, under normal circumstances, the village health teams are supposed to work harmoniously within their villages to refer clients to facilities and carry out health education talks. (In-charge at a health facility in one participating district)

Recruitment of traditional female sex educators/aunties as interpersonal champions

Traditional female sex educators or aunties, popularly known as Ssengas, were identified as influential and credible sources of information for adolescent girls. Therefore, recruitment of aunties as interpersonal communication champions was highlighted as vital. On the contrary, some program records showed that adolescents were alienated from interacting with aunties on sexuality issues because they were not updated with fast-growing technological advancements. For example, the aunties’ inability to use advanced communication over internet-based social media (popularly used for teenage communication) distances them when discussing sexuality issues with adolescents. It was noted that some aunties do not keep secrets, which affects confidentiality in discussions of sexuality issues with adolescents. Therefore, equipping aunties with updated communication skills and effective supervision is pertinent. This is illustrated in the quotes drawn from ARC program documents showing qualitative assessment and implementation research findings.

Facebook has kept the aunties far from the adolescent. So, the adolescent learns most sexual issues from the internet because the aunt does not know what is trendy. Examples in her talk are archaic and unconnected to the present internet exposure. (Female adolescent, Jinja district)

Aunties also do not keep confidentiality and speak loud about adolescent secrets at boreholes. (Female adolescent, Mayuge district)

Linking income-generating activities to health education

Linking health education with livelihood and income-generating activities motivates adolescents to seek health services. Livelihood interventions were identified as important in providing alternative choices for female adolescents to receive income rather than depend on male partners [36]. This evidence guided program adaptation in integrating skills for income-generating activities including farming, kneading, weaving and hairdressing to support female adolescents’ livelihoods. This is illustrated by the quote below as drawn from the ARC program documents showing qualitative assessment and implementation research findings.

Adolescents look towards employing themselves by waking up early to go cultivate or make bricks to keep them busy so that they do not run after sex. If properly done they can earn enough to pay for their school fees. (Female adolescent, Kamuli district)

Values clarification tool used to demystify gender and misconceptions about contraception use

The use of Obulamu? Values Clarification Tool (Fig. 5) was important in highlighting gender issues and addressing myths and misconceptions on contraception use during interpersonal communication discussions. Although the program trained champions on the use of the Values Clarification Tool, for maximum benefit the use of the tool was only as effective as the user’s ability to correctly apply the tool and avoid personal bias. The Values Clarification Tool assisted champions to re-evaluate their perceptions of adolescent sexual and reproductive health in regard to seeking information on sexuality and contraception services. However, the champions’ ability to effectively use the Values Clarification Tool in their routine work was not detailed in ARC documentation [42]. Nonetheless, the Values Clarification Tool can be adapted in adolescent health programming and revised to reflect evolving/emerging issues and needs that apply to context.

Fig. 5.

Obulamu? Values Clarification Tool used by CHC for the ARC program.

Discussion

CHC’s planned ARC activities including training and deploying community champions to conduct health education talks, community edutainment events and mass-media message placements were successfully implemented in collaboration with service partners and the community in Uganda. Interpersonal communication complemented by mass media was effective in reaching and empowering adolescents with health information to make informed choices. Adolescent interaction with the community champions helped them understand contraception use information to facilitate behavior change and service uptake. The ARC program used theoretical frameworks to guide implementation (including the Social Ecological Model for audience segmentation), and Empowerment theory for community empowerment through collaborative partnerships and shared decision making. Through its instituted measures of assessing the progress of activity implementation, we noted that the trained/oriented community champions were able to execute their interpersonal communication activities. However, many champions were not confident about all health topics, which indicated that they needed additional refresher training. Our evaluation found that the use of mass-media messaging without targeted placement affected program reach to the intended target audiences. National political elections disrupted health communication interventions because they were mistaken as partisan. Targeting a large number of community champions was difficult for follow-up and reporting, therefore limiting the number to a manageable size and working with existing community structures is important. Effective evaluation of program reach was challenged by the limited data disaggregation in most ARC reports, which did not indicate program reach to the various sub-groups of the target population.

Our findings reflect those of other studies. Collaborative partnerships for community health and development provide opportunities for studying and contributing to empowerment [43]. Previous studies found that shared decision making empowers individuals to exercise some control over local services and recognize the importance of matching program delivery to local needs, preferences and cultural norms [44, 45]. Using message-effect theories to design messages for target audiences can enhance the probability of a campaign’s success [46]. Mass-media messaging by radio has been found to be a common method for reaching adolescents if matched with targeted message placement. UBOS (2018) found that radio is the most common source for family-planning messages in Uganda since 57% of girls and 55% of boys reported hearing a family-planning message on the radio. Women who view more family-planning messages on television, radio, and print media were found to be more likely than those who see fewer messages to use contraceptives [47]. Multi-media campaigns are more likely to result in greater impact since messages are mutually reinforced [48]. Communication channels including media and interpersonal communication should be selected and used in a way that reaches a high percentage of the target population multiple times over a given period, because the greater a campaign’s reach (exposure), the greater the behavior change [46, 49, 50]. Appropriately trained community health workers can effectively support the health system by improving the community-facility linkage [51, 52]. Health authorities should lead efforts to define and endorse community health workers’ roles while providing strategies for training, supervision, remuneration, recognition, career progression and quality assurance [12, 52].

Lessons and implications for future programming

Several lessons emerged from the findings of this evaluation that have implications on future adolescent health programming. First, it was noted that recruitment of champions with a greater influence on the target population was key. Adequately training and equipping selected champions with skills and updated information was vital for reaching adolescents. Therefore, development practitioners should adopt an integrated program-design approach to work collaboratively and harmoniously in providing continuous refresher training and champion deployment for effective implementation [12, 52, 53]. Harmonization of partner-level champion-led efforts should be driven by the government through the Ministry of Health. Second, linking income-generating activities to health promotion programs is pertinent for reaching adolescents. Livelihood interventions provide alternative options for adolescents to generate income. Sheheli (2012) found that building women’s individual incomes is essential for eliminating poverty since it helps to build a base for social change. Income-generating activities change the livelihood of the poor including living conditions, housing, nutrition, savings, dress, medical treatment, health, sanitation, liberalization and education [54, 55]. Third, program developers should adopt activity implementation tools that have been tested and used by other adolescent health programs. This will help new programs to scale-up best practices of existing projects for continuity toward sustainability. Overall, this evaluation indicates the pertinent need to periodically evaluate existing adolescent health programs to draw additional lessons for implementation and scale-up. Studies indicate that relatively few communication interventions have been subjected to rigorous evaluation [56, 57]. Most adolescent programs that exist do not systematically document their activities, implying that they cannot be evaluated objectively [56]. Few studies address the costs and cost-effectiveness of mass communication programming, leaving funders and policymakers without the data necessary to determine which intervention strategies offer the greatest “bang for the buck” [57].

Study limitations

This study has some limitations. Our evaluation was conducted retrospectively based on program records, and staff information was subject to recall bias. However, using multiple data sources helped in tracing the implementation footsteps to objectively evaluate the program. The study used adolescent self-report data based on quantitative and qualitative program records, which are subject to bias. However, this self-report data provided practical insights into areas for strengthening scale-up of adolescent health programs.

Conclusion

Evaluation of CHC’s ARC program showed that the campaign’s implementation was successful in the Ugandan context. Effective collaboration and partnerships with policy and development partners and community members in intervention implementation were critical for ownership and continuing gains. However, full program monitoring and evaluation requires detailed data disaggregation of program reach to various sub-groups of the target population and needs to be strengthened in scale-up programs. Strategies for monitoring and evaluation of adolescent health programs should be collaboratively developed during program inception and design for proper accountability of health promotion and communication program efforts.

Funding

Funding for this study was provided by Maastricht University in the Netherlands, and data collection was supported by United States Agency for International Development (USAID) through, Communication for Healthy Communities (CHC) program RFA-617-13-000001 under FHI 360.

Conflict of interest statement

None declared.