Digital self-care for improved access to family planning and reproductive health services among adolescents in Rwanda: preliminary findings from a pilot study of CyberRwanda.

Adolescents experience significant barriers, including stigma and discrimination, to accessing voluntary family planning and reproductive health (FP/RH) services in Rwanda. Self-care interventions have been shown to reduce social barriers to FP/RH care, but little is known about the effectiveness of digital self-care for adolescents, particularly in low-resource settings. This paper presents findings from a pilot study of CyberRwanda, a digital self-care intervention providing comprehensive sexuality education and confidential online ordering of contraceptives for school-aged youth in Rwanda through a rights-based approach. A mixed-methods pilot study was conducted from November 2019 to February 2020 to assess feasibility, acceptability, and engagement and to inform a future impact evaluation. Surveys were administered to a random sample of 158 students aged 12-19 years in three secondary schools. In-depth interviews were conducted with students, parents, teachers, pharmacists, district-level administrators, and youth centre staff. Descriptive statistics were calculated and qualitative data were analyzed using a thematic coding approach. One hundred and fifty-eight surveys and 28 interviews were conducted. Results revealed high demand for CyberRwanda in schools. Students were interested in engaging with the program and found the FP/RH content relevant to their needs. However, few purchased contraceptive products through the online ordering system. There are preliminary indications that CyberRwanda may improve access to FP/RH information. An extended implementation period and further research are needed to measure the long-term impacts of the program and evaluate whether this digital self-care intervention can increase uptake of contraceptive methods and reduce adolescent pregnancy among school-aged youth.

Introduction

Adolescent girls in Rwanda confront numerous barriers to accessing family planning and reproductive health (FP/RH) services, resulting in a large unmet need for modern contraceptives and high rates of unintended pregnancy.[1-4] Although access to services has improved considerably over the past several years,[5] meeting the FP/RH needs of youth remains a challenge due to various factors including prohibitive beliefs and norms, and behavioural biases.[3,6] Data from the 2019–2020 Demographic and Health Survey indicate that more than 15% of girls aged 19 have begun childbearing and 7.6% of girls aged 15–19 have an unmet need for family planning. The adolescent fertility rate in Rwanda remains largely unchanged over the last decade, from 40 live births to 39 live births (per 1000 among 15–19-year-old women).[2,7] Addressing early childbearing is critical given the significant social, educational, and economic consequences for young women as well as the severe health effects including higher risk of obstetric complications, infection, unsafe abortion, gender-based violence, and premature maternal death.[8-10]

Stigmatisation of FP/RH and discrimination from providers are frequently reported by young women and girls as reasons for not seeking services.[1,11-13] Indeed, provider bias has been shown to be especially harmful to unmarried, nulliparous, and school-aged youth across sub-Saharan Africa and in Rwanda in particular. These groups report experiencing fear, shame, and judgement when purchasing contraceptive products and consequently avoid pharmacies and healthcare facilities for FP/RH.[1,14] Thus, in order to strengthen current care models and increase access for youth, reducing discrimination and improving the user experience is essential.[15]

Digital self-care, which aims to promote and maintain health with or without the assistance of a healthcare provider, has been widely adopted to facilitate access to high-quality, patient-centred health services and information.[16,17] This approach was developed with the goal of increasing autonomy and self-efficacy, preserving anonymity, and improving linkage to and continuity of care.[16] Previous studies and guidance from the World Health Organization support the potential of digital self-care approaches to improve FP/RH. In particular, digital approaches are suited for the provision of autonomous and private information on often-taboo topics as well as direct-to-consumer services for confidential and streamlined ordering of FP/RH products from local healthcare providers.[16,18] Although non-digital self-care can improve mechanisms for accessing self-managed contraceptives and provide additional access to other FP/RH products,[19-21] few studies have investigated whether digital self-care interventions can overcome the unique barriers faced by adolescents in low and middle-income countries.

This paper explores how the CyberRwanda program, a digital self-care intervention for urban and peri-urban youth in Rwanda, addresses social and structural barriers to potentially improve access to FP/RH information, uptake of modern contraceptive methods, and reduce pregnancy and early childbearing among adolescents in Rwanda. Designed in partnership with over 1000 youth and their communities, CyberRwanda provides comprehensive sexuality education through age-appropriate storytelling content about FP/RH and streamlined access to contraception and FP/RH products at nearby pharmacies through a mobile ordering platform. We present results from a mixed-methods pilot study of two CyberRwanda implementation models and discuss steps for future research and action.

Methods

Pilot study overview

The CyberRwanda program was piloted in three schools and one youth centre from September to November 2019 (Figure 1). A mixed-methods study was conducted from November 2019 to February 2020 to explore whether two CyberRwanda implementation models (described below) were feasible and acceptable to stakeholders, to quantify youth engagement with content and features, and to inform a future impact evaluation assessing the effectiveness of the program. Data collection consisted of a cross-sectional quantitative survey with youth in schools and in-depth interviews (IDIs) with youth, pharmacists, teachers, district-level administrators, parents, and youth centre staff to evaluate the feasibility, acceptability, relevance of the educational content, and willingness of youth to engage with the platform. Ethical approval for the pilot study and impact evaluation was obtained by the Committee for Protection of Human Subjects at the University of California, Berkeley (Protocol 2019-04-12145, approved October 18, 2019), and the Rwanda National Ethics Committee (IRB 00001497, approved August 27, 2019).

Figure 1.

CyberRwanda timeline

CyberRwanda program

CyberRwanda is a direct-to-consumer platform that delivers age-appropriate information on health and employment, and linkage to pharmacy services for adolescents. The platform was developed through a previously described human-centred design (HCD) process with Rwandan adolescents, their parents, teachers, and healthcare providers.[22] The HCD process was driven by a rights-based approach and included meaningful participation of adolescents and their communities to ensure equitable access to the final intervention and meaningful, context-specific support for youth to prevent unplanned pregnancies[23] (Table 1).

Table 1.

CyberRwanda design methods and intervention using a human rights approach to self-care for sexual and reproductive health and rights[16]

Human Rights Standards[16]	Design phase methods and findings	Final intervention design components
The right to health, including availability, accessibility, acceptability and quality of information, goods, and services	CyberRwanda was designed in collaboration with more than 1000 youth, providers, teachers, and community members using a participatory, youth-driven design approach23 to identify key challenges for youth and address barriers to quality care.	All content on the platform is age-appropriate and aligned with national and international evidence-based practices on comprehensive sexuality education.
	The final CyberRwanda product was prototyped with adolescents with diverse levels of literacy, digital access, and fluency to maximise accessibility.	The web-comic and FAQ content is designed specifically for those with both low and high literacy levels.
The right to participation	The design research, prototyping, and pilot phases aimed to maximise participation in CyberRwanda for both in- and out-of-school youth and ensure that the STORIES, LEARN, and SHOP functions were accessible and understandable to all youth regardless of access to a mobile device.	Youth are provided with information to make their own FP/RH related decisions. The SHOP offers details on health products, side-effects, and uses. The STORIES component models conversations with health providers and parents to support youth in navigating challenging conversations about their health.
		Pharmacists are trained to provide youth-friendly care and identify provider bias to improve quality of care.
The right to equality and nondiscrimination	Participants in the design process were sampled to ensure diverse representation with regard to socio-economic status, gender, age, educational status, and social vulnerabilities (former sex workers, insecurely housed youth, domestic workers).	To reach youth with and without access to technology, CyberRwanda is offered on tablets installed in schools and youth centres, and on personal devices.
		To reach youth who may not be in school, CyberRwanda is offered in youth centres.
		To ensure all content is non-discriminatory, content was reviewed by local and global gender, health, and other experts, validated by youth, and approved by the Ministry of Education and the Health Communications Technical Working Groups.
The right to information	A key finding from the formative and design research processes was that youth lack access to accessible, accurate, and clear FP/RH information.	Youth set the priority topic areas, suggested the FAQs covered in LEARN and co-designed how the information is delivered. The information provided was tested and validated with youth to help ensure that it is understandable and meets their needs.
The right to informed decision-making	Formative and design research revealed main barriers to care including fear of judgment and method refusal by providers. Providers were reported to act as “gatekeepers” to accessing methods and to hold non-evidence-based beliefs on the side-effects and safety of contraceptive methods for adolescents.	The SHOP is a direct-to-consumer feature of the platform that was designed to put accurate information and healthcare products directly in the hands of youth and reduce barriers associated with gatekeeping and fear of judgement. Youth are guided when medically necessary to an additional directory of medical services to access other services e.g. pregnancy testing or HIV testing.
The right to privacy and confidentiality	Privacy and confidentiality were identified during the design phase as priority areas for youth. In early CyberRwanda prototypes, clinics were originally intended as the sole site for CyberRwanda service linkages. During subsequent research and prototyping, the focus shifted to pharmacies in response to youth preferences for more private, confidential care.	CyberRwanda enables private and confidential ordering of potentially taboo or stigmatised products. From order to pick-up, the service is designed so that only the young person knows the product they ordered and never needs to discuss with a pharmacist.
		All data from the CyberRwanda platform is gathered and stored in compliance with General Data Protection Regulation guidelines. No identifiable data are collected from users and a privacy policy is available in Kinyarwanda and English in non-technical language.

The platform includes three features (Figure 2):

Figure 2.

CyberRwanda platform features and implementation models

STORIES: Fun and engaging narrative content in a web-comic format to engage all literacy levels in Kinyarwanda and English. Topics include career planning, goal-setting, puberty, contraception, gender-based violence, relationships, and consent.

LEARN: A robust library of questions with over 200 FAQs and a directory to help youth locate both public and private health facilities and pharmacies in their communities.

SHOP: Youth can discreetly and privately order, purchase, and pick-up health products including condoms, emergency contraceptives, oral contraceptive pills, pads, ibuprofen, and paracetamol at the CyberRwanda pharmacy of their choice. Pricing is transparent on the platform and users can choose from local pharmacies in the network.

A network of private CyberRwanda pharmacists and their staff are trained to support the provision of high-quality, youth-friendly care. Pharmacies in the network can add and update health products in the SHOP section of the platform. Youth seeking products can then choose from a local pharmacist in the network when placing orders, enabling health professionals to build relationships with young people in their communities.

Implementation models

CyberRwanda is delivered on networked tablets through two implementation models: self-service and facilitated. In the self-service model, schools and youth centres receive tablets, hotspots, marketing materials, and training on how to set up tablets in select in-school and youth centre locations for individual use. Youth are selected as school/youth centre ambassadors and trained on how to use the platform and how to support their peers on better understanding CyberRwanda topics. In the facilitated model, the same implementation materials are provided as the self-service model with the addition of peer facilitators. The peer facilitators guide youth through structured sessions outlined in a CyberRwanda activity booklet. School/youth centre ambassadors, peer facilitators, lead teachers, and youth centre staff receive a 1.5-day training on how to use CyberRwanda and on fundamental FP/RH concepts. Peer facilitators receive additional day-long training on the facilitation of the activity booklet. In both implementation models, youth have the opportunity to order FP/RH and other products through the platform from participating CyberRwanda pharmacies, using the tablets at school or the youth centre or their own devices.

Theoretical framework underlying the CyberRwanda program and evaluation

The theory of planned behaviour (TPB)[24] was used to inform the hypothesised impact pathway and to select the primary and secondary outcomes for the pilot study and future impact evaluation. The TPB states that intention is central to behaviour change, and that the primary influencers of intention are knowledge/beliefs and perceived behavioural control, both of which can shift subjective norms and thus influence intentions related to a particular behaviour. Actual control (juxtaposed with perceived behavioural control) can directly influence a behaviour by, for example, improving access by removing structural barriers. We hypothesise that CyberRwanda’s tailored narrative and content can influence behaviour change by increasing knowledge, perceived behavioural control, and access to FP/RH products through online ordering and integration with youth-friendly pharmacies.

Pilot setting and implementation

The CyberRwanda pilot was conducted in three districts. Study sites included three secondary schools in the Gasabo and Nyagatare districts and one youth centre in Nyarugenge district. One school received the self-service model and two schools and the youth centre received the facilitated model. Prior to implementation, parents, head teachers, and administrators in each school were consulted to assess the most appropriate way to introduce CyberRwanda and to gather input on the educational content and implementation models.

The two pharmacies closest to each pilot school and the youth centre were identified and asked to participate in the program. Pharmacies were invited to a one-day training on the CyberRwanda intervention and youth-friendly FP/RH, including safe and confidential practices for adolescents. The CyberRwanda team communicated regularly with the network of pharmacists after implementation to receive feedback and address technical and logistical issues.

Sampling and data collection

Students between 12 and 19 years of age in levels S2 through S4 in pilot schools were eligible to participate in quantitative surveys. A target sample size of 150 students was determined based on school administrator data estimating 76 eligible students per school, an approximate 1:1 ratio of males to females, and ∼33% potential absences (50/76 students present) on the day of data collection. Thus, to reach this target sample size, 75 students meeting the eligibility criteria in each school were randomly selected from school rosters. Selected students who were <18 years of age provided written informed consent from parents or guardians and informed assent. Consent was obtained from students ≥18 years. Surveys were conducted on tablets in Kinyarwanda by research assistants using Qualtrics offline survey software.[25] Surveys included questions on sociodemographic characteristics, knowledge and social norms related to FP/RH, gender roles, gender-based violence, fertility intentions, use of contraceptives, communication with partners, sexual behaviour, childbearing, future aspirations and goal-setting, and attitudes and intentions toward school and employment.

A convenience sample of students, parents, school staff, pharmacists, district officials, and youth centre staff were recruited for IDIs to assess the feasibility and acceptability of CyberRwanda in schools, youth centres, and pharmacies. To ensure that interviews were conducted with students who had experience using CyberRwanda, students were selected purposively based on their reported interaction with the CyberRwanda platform in the quantitative survey. IDIs were conducted in Kinyarwanda using a semi-structured interview guide (see Supplementary Material) developed to assess various components of the CyberRwanda program. Questions explored knowledge and perceptions of CyberRwanda, implementation challenges and successes, challenges for youth in the community, and pharmacy experiences. Interviews were approximately 45 minutes in duration and audio recorded. Recordings were transcribed verbatim and translated into English for coding and analysis.

Data analysis

The pilot study had three primary outcomes: engagement, acceptability, and feasibility of the CyberRwanda platform (Table 2). Descriptive analyses of the quantitative survey data were conducted using R Studio version 4.0.4 to assess engagement with CyberRwanda.[26] IDI data were analyzed inductively using a thematic analysis coding approach and focused on self-reported engagement, and acceptability and feasibility of the platform.[27] Interview excerpts were coded and extracted from transcripts by one member of the research team. Emerging concepts were grouped in a table and main themes were generated, distilled, and reviewed by the Rwanda-based study team.

Table 2.

Primary outcomes for CyberRwanda pilot study

Primary outcome	Indicator	Data source
Acceptability	Suitability of CyberRwanda for youth, including whether the platform content was relevant to youth needs and interests	In-depth interviews
	Support for CyberRwanda by school, youth centre, and pharmacy stakeholders
Feasibility	Ability for integrate CyberRwanda in schools and youth centres	In-depth interviews
	Capacity to support CyberRwanda implementation
Engagement	The proportion of students in participating pilot schools who had heard of CyberRwanda platform	Quantitative surveys; in-depth interviews
	The proportion of students in participating pilot schools who had used CyberRwanda (overall, and by-product feature)
	Student interest and demand for CyberRwanda

In addition to measuring the primary outcomes, the pilot also sought to describe the demographics of the student population and to conduct an exploratory assessment of the impact evaluation primary outcomes (modern method use, initiation of childbearing, and HIV testing).

Results

Study population

A total of 158 quantitative surveys were conducted. Of the 225 randomly selected students, 67 were absent on the day of data collection (29.7%). All students who were present consented or assented to participate. Table 3 presents the demographics of the study population who participated in surveys and the prevalence of the primary study outcomes to be assessed during the impact evaluation (current use of a contraceptive method, initiation of childbearing, and ever having tested for HIV). Overall, 19 (12.0%) students reported that they were sexually active and were asked follow-up questions about contraceptive use. Eleven (7.0%) reported using a modern contraceptive method, with eight (72.7%) using condoms, two (18.2%) using the Standard Days Method, and one (0.09%) using a diaphragm. None of the females reported a previous or current pregnancy or live birth. Seventy students (44.3%) reported ever having tested for HIV, of which seven (4.4%) reported ever having used an HIV self-test.

Table 3.

Demographics of students completing quantitative surveys, Rwanda, 2019–2020

	Facilitated (N = 100)	Self-service (N = 58)	Overall (N = 158)
Age
Mean (SD)	15.8 (2.09)	15.9 (1.44)	15.8 (1.87)
Median [Min, Max]	16.0 [12.0, 19.0]	16.0 [12.0, 19.0]	16.0 [12.0, 19.0]
Sex
Female	54 (54.0%)	35 (60.3%)	89 (56.3%)
Male	46 (46.0%)	23 (39.7%)	69 (43.7%)
School level
S2	29 (29.0%)	33 (56.9%)	62 (39.2%)
S3	31 (31.0%)	19 (32.8%)	50 (31.6%)
S4	40 (40.0%)	6 (10.3%)	46 (29.1%)
Sexually active	17 (17.0%)	2 (3.4%)	19 (12.0%)
Using modern methoda,b	10 (10.0%)	1 (1.7%)	11 (7.0%)
Ever been pregnanta	0 (0.0%)	0 (0.0%)	0 (0.0%)
Ever given birth*	0.0%	0.0%	0.0%
Ever tested for HIV*	49 (49.0%)	21 (36.2%)	70 (44.3%)
Ever used HIV self-test	2 (2.0%)	5 (8.6%)	7 (4.4%)

Primary outcomes for the CyberRwanda impact evaluation.

Assessed among sexually active students only.

Twenty-eight in-depth interviews were conducted with stakeholders (96.5% response rate). Ten of the interviews were with students, three with parents, four with pharmacists (from half of participating pharmacies), six with teachers, one with youth centre staff, and four with district-level administrators.

Feasibility and acceptability of CyberRwanda to students

When asked about perceptions of the CyberRwanda content, many students shared that the information on FP/RH and employment was applicable to their experience as youth in Rwanda and relevant to their daily lives.

“There are various things that motivated me to use the CyberRwanda website, but the most important include getting useful information about reproductive health and life skills in preventing against unplanned pregnancies, sexually transmitted diseases and HIV/AIDS, information on how to prepare for my future, employment opportunities, how to behave in society, and how to use contraceptive methods like condoms and pills.” (Male student, 16, S4)

In particular, students reported a keen interest in the STORIES and LEARN features, where they could explore sensitive questions that they may not feel comfortable asking adults. One student shared how he used LEARN to understand how to prevent HIV/STI infection and unplanned pregnancies.

“I learned that when you are facing a challenge, you can browse the FAQs [in LEARN feature] and ask a question and then receive an immediate response. You can even ask a question you cannot dare ask your parents … I learned how to prevent against sexually transmitted diseases like HIV/AIDS, gonorrhea, syphilis etc. The information I got from CyberRwanda told me that when I fail to abstain myself, I can use a condom. CyberRwanda boosted our knowledge about how to prevent against unplanned pregnancies. Before it was difficult to get a condom, but through CyberRwanda things are easy. I also learned how I can order hygienic products and pills easily. There are products a woman can use when she is on her menstrual period. There are also contraceptive pills a girl can use when she has had unsafe sex or when she is impregnated.” (Male student, 17, S2)

Another student discussed how the STORIES and LEARN features helped clarify some of his misconceptions about pregnancy and emergency contraception. He noted that his experience with the platform allowed him to become informed on these topics that he otherwise would not have known about or believed from other sources:

“One thing I learned from the CyberRwanda program is about the emergency contraceptive pill. I didn’t know this, I heard about it through my Biology course and didn’t even believe in it. I had a perception that when a girl gets pregnant, there’s nothing to do other than wait to have a child. Or when a girl has forgotten to take emergency contraceptive after 72 hours after unsafe sex, I thought that that it is over; if you attempt to [end a pregnancy], you can die. I learned all [of this] from the CyberRwanda website.” (Male student, 18, S4)

Students appreciated that CyberRwanda is confidential and that questions and information about SHOP orders are private. They described feeling judged when going to a shop to buy condoms, and shared that CyberRwanda provided an opportunity to order products in a discreet manner without being seen by other community members:

“The teacher cannot tell you where to find condoms and pills, but the CyberRwanda program does everything in high confidentiality. A teacher can teach you about using condoms and the emergency pill but doesn’t tell you where to find it, but CyberRwanda tells you where to find those items. As young people, we fear being seen by a neighbor at the shop when you have to buy a contraceptive, you are shy that you are judged of what you are going to use the condom for, but CyberRwanda does it in confidential way.” (Male student, 18, S4)

Feasibility and acceptability of CyberRwanda to school staff and other stakeholders

Teachers reported that students were excited about the program, however sentiments among school staff and teachers were mixed. Some teachers were in favour of the program and found it useful in supplementing their existing biology curriculum that includes modules on FP/RH. However, other teachers and staff were more hesitant to engage with and support the program. One CyberRwanda peer facilitator noted that some teachers felt that the program promoted sexual activity among youth by providing information about FP/RH and how to access contraceptives:

“Our fellow teachers said that the program is mobilizing youth to have sex because it is providing them with information of where and how to get condoms and pills to help them have sex. On the other hand, the students from the CyberRwanda club were so interested and asked so many questions that we didn’t have time to respond and serve all of them because their demand was so big.” (Student, CyberRwanda peer facilitator)

Other teachers were reluctant to embrace the program due to the disruption of schedules and class timetables, and fear that the tablets serve as a distraction to students. In light of these challenges, one head teacher proposed engaging teachers and school staff in a workshop to discuss CyberRwanda and encourage collaboration within the school and among staff:

“The second challenge I am thinking about is mindset and resistance for leaders to adopt the new system in place. For example, you can hear a school leader saying, ‘this has come to disturb us and be added on the timetable’. The other challenge is about parents’ mindset and resistance that their children are going to shift their attention to tablets and also the tablets can be destroyed in a few months … The school leadership need a workshop to talk deeply about the project, the processes and procedures of engagement and collaboration.” (Head teacher)

Other stakeholders also noted that some parents do not approve of the content of the program, particularly those with lower levels of educational attainment:

“For parents, there are those who appreciate it especially those who went to school but there are others who don’t feel it’s an ideal program because they think it will make their children have bad behaviors.” (District Director of Education)

“There are some parents who think that the program can mislead their children. So it will be a matter of educating parents as well as children. However, the understanding is a process; the role of school leadership who are involved in the education sector will play a key role in collaborating with CyberRwanda in attaining its objectives.” (Director of Health)

Pharmacists were generally very supportive of the program but recognised some limitations and areas for improvement. For example, more than one pharmacist mentioned the need to train all staff working in the pharmacy, rather than just the head pharmacist, to ensure that all staff were prepared to serve youth ordering through the CyberRwanda platform. Additionally, there was some recognition that youth need training, education, and mobilisation to build the confidence needed to recognise their needs and request services related to FP/RH from community pharmacists:

“To be effective, the program needs to intensify the training of youth and create confidence in requesting the needed services from pharmacies and health departments.” (Pharmacist)

Student engagement with the CyberRwanda platform & content

Teachers in the pilot schools reported high demand for CyberRwanda. One CyberRwanda peer facilitator felt that the student demand for the club was driven by their interest in the FP/RH information provided by CyberRwanda. The facilitator suggested that students feel more comfortable engaging with these sensitive topics with teachers and facilitators rather than with parents:

“The first time CyberRwanda came here, a large number of students were curious to ask the questions because some children fear asking questions related to life skills to parents and prefer to ask those questions to the teachers.” (Student, CyberRwanda peer facilitator)

Students also expressed that the program was beneficial and hoped that it would continue to be offered in their school and other schools after the pilot:

“My school should continue to receive this CyberRwanda program to help children who are not able to self-abstain access contraceptives. Other schools should also benefit from this program because students experience the same challenges.” (Female student, 14, S2)

Survey data indicated that overall, nearly all students had heard of CyberRwanda (97.5%). Of those, 63.6% reported using CyberRwanda during the pilot period. More than 62% of students who used the platform stated they used it to learn about contraceptives, 29.6% reported using it to help plan for their future, and 18.4% said they used it for ordering a health product (Table 4).

Table 4.

Knowledge and use of CyberRwanda by students, overall and stratified by implementation model (facilitated or self-service)

Engagement type	Facilitated (N = 100)	Self-service (N = 58)	Overall (N = 158)
Heard of CyberRwanda	96 (96.0%)	58 (100%)	154 (97.5%)
	Facilitated (N = 96)	Self-service (N = 58)	Overall (N = 154)
Used CyberRwanda	45 (46.9%)	53 (91.4%)	98 (63.6%)
	Facilitated (N = 45)	Self-service (N = 53)	Overall (N = 98)
Used to learn about contraceptives	22 (48.9%)	39 (73.6%)	61 (62.2%)
Used to order products	8 (17.8%)	10 (18.9%)	18 (18.4%)
Used to plan for their future	11 (24.4%)	18 (34.0%)	29 (29.6%)

There were notable and unexpected differences in use of CyberRwanda by type of implementation model. A higher proportion of students reported interacting with the CyberRwanda platform at the self-service school as compared to the facilitated schools. This finding was further explored during interviews with students and school administrators and revealed the need for improved collaboration with teachers and standardised instructions to reduce gatekeeping of tablets to ensure students have consistent and regular access to CyberRwanda in both models.

Survey data also suggest that students using both implementation models talked with others about CyberRwanda, another indicator of interest and engagement with the platform. Overall, 103 (65.2%) students reported talking about the CyberRwanda program with friends or family; 37 (23.4%) discussed the program with peers at school and 63 (39.9%) with peers outside of school. Thirty-one students (19.6%) had told their parents about the program.

Engagement with the CyberRwanda SHOP feature

Although interest and engagement with CyberRwanda were high, there were few pharmacy purchases made through the SHOP feature. Of the 158 students surveyed, 18 (11.4%) reported using the online ordering system, and six (3.8%) indicated that they had purchased a product, suggesting that students may have explored the shopping and ordering system without actually ordering any products. Of students that did purchase a product through the online pharmacy, only one bought a modern contraceptive method (condom). Other purchases included paracetamol and pads. When asked about the ease of use of the online ordering system, four of the six students who purchased items reported that the ordering process was easy or very easy, and two students reported that it was difficult to use.

Interviews confirmed that students were aware and interested in the SHOP feature and knowledgeable about how it functions. However, when asked whether they purchased any products, students reported that they had not made any orders, despite their excitement about the confidential ordering process:

“In this program, I was interested by the part called SHOP. You can order contraceptive products using this program and get the products without the people knowing what you ordered …  I like this part so much more than other parts, but I didn’t order anything.” (Female student, 14, S2)

Stakeholders were asked about the reasons why students were not purchasing more products with the SHOP feature, specifically contraceptives and other FP/RH products. They suggested that the low usage of the ordering feature may be related to fear or embarrassment. They described shame and stigma as deeply rooted social barriers that would be challenging to overcome, despite students knowing that the ordering process is confidential and highlighting that they appreciated that feature of the platform:

“I talked with students and they told me that for a young girl of their age they feel shame to even buy a pad in the pharmacy, [not to mention] buying a condom or pills.” (Student, CyberRwanda peer facilitator)

“Some of the questions they [students] asked are … If the teacher finds the condom in your pocket, can he punish you? How can you proceed when you experience shame while buying contraceptive products?” (Head Teacher)

Some indicated that a lack of financial means may also pose a barrier to purchasing products through the SHOP feature. Stakeholders noted that students typically have to ask their parents for money and would have to explain and justify their purchases.

“My worry is about the child from secondary schools who don’t have financial resources. Where will the youth get money to pay for the products from [the] pharmacy? 100, 200 or 500 francs, where will they get it from? Will they request the money from their parents? What are they going to tell their parents that money is for?” (Director of Health)

Others suggested that those who did order products through the online ordering system may have been reluctant to go to the pharmacy to pick them up. One CyberRwanda facilitator noted that youth may have had curiosity and interest in testing the feature to see how it worked, however did not have the intention of completing the pick-up process:

“When I asked them what they have learned, they responded saying that they tried to order some products and unfortunately, they did not go pick the order at the pharmacy … As you know, teenagers are stubborn and curious, they do so in trying without any intention of buying. The evidence is that some pharmacists told me that they received the orders from our school but the students did not come to pick [up] their orders.” (Student, CyberRwanda peer facilitator)

However, one adolescent at the youth centre who was not surveyed did complete order successfully, according to the youth centre coordinator. He shared that she expressed being pleased with the experience and felt that the SHOP feature worked well:

“One day, I received a testimony from a girl who ordered pads online to test the reliability of the program and the service it is offering … she got it from the nearby pharmacy and when she came back she showed it to me confirming that the program works very well. She added that when she got there, the pharmacist took the sealed pads and gave them to her after payment. No one in the pharmacy noticed what happened since there were many customers.” (Youth Center Coordinator)

Discussion

The findings from this mixed-methods pilot study suggest that the CyberRwanda program, a digital self-care intervention designed for and by youth is feasible and acceptable to students, teachers, pharmacists, and other community members. Initial levels of engagement with the platform indicate that CyberRwanda has the potential to improve FP/RH knowledge among school-aged adolescents in Rwanda through increased access to youth-friendly information. Further assessment and a longer implementation period are now needed to determine whether the program can be effective in increasing contraceptive uptake, reducing early pregnancy, and increasing HIV testing.

Acceptability and feasibility of CyberRwanda

High demand and enthusiasm for the CyberRwanda program were found among youth in schools that received the intervention, with students reporting that the content was relevant to their lives and daily needs. Interest and buy-in were also reported from youth centre staff, teachers, pharmacists, community members, and youth centre staff. Stakeholders raised issues that may need additional consideration in preparation for the future impact evaluation, including ensuring that students are mobilised and have the confidence to use the online pharmacy feature, increasing the engagement and training sessions with parents, teachers, and school staff who may have reservations about implementing CyberRwanda in schools, and working across the community to help decrease fear, shame, and stigma that many adolescents experience related to FP/RH, even for less sensitive and stigmatised products such as sanitary pads.

Engagement with the CyberRwanda platform

Students were interested in receiving information about FP/RH and reported high usage of the STORIES and LEARN features. Nearly 40% used CyberRwanda to learn about contraceptives. IDIs revealed that ordering contraceptive products through the confidential SHOP feature was also of great interest to students; however, actual utilisation of this feature was low. Although students described their appreciation of the confidential nature of the SHOP feature and intention to use the platform, there was reluctance to purchase products. Possible reasons highlighted by stakeholders included fear and shame, as well as financial barriers. Despite not collecting or storing any individual user data, there may also have been insufficient time to establish trust and build confidence in using the platform given the short duration of implementation for the pilot study. Low uptake of the SHOP feature could potentially be explained by other reasons, such as the low number of direct-to-consumer platforms available at the time of the pilot and the low familiarity with online ordering among the target audience. We found that many students had no previous experience with online ordering, which is consistent with other evidence[28] and suggests that although experience with direct-to-consumer platforms is increasing, usability testing, especially with users with low digital fluency, should be prioritised. However, despite low usage during the pilot, interest and appreciation of private and confidential online ordering as well as shared positive experiences of those who did successfully order through the SHOP, suggest that discreet shopping for contraceptive products shows promise in increasing access to essential FP/RH products for youth.

Post-pilot adaptions to the platform and implementation model

Based on our findings, multiple aspects of the CyberRwanda platform are being iterated on to address some of the issues raised by stakeholders as potential barriers to engagement with and uptake of the online pharmacy. This includes further simplifying the ordering user experience, using more visual guides throughout the digital interface, and developing onboarding videos that describe the site and how to use the SHOP. A free mystery gift product was added to the SHOP (notebooks, pens, highlighters) to provide users with the opportunity to familiarise themselves with the feature through ordering non-health related products. Discounted products were also added to the SHOP (condoms and emergency contraception) to reduce cost barriers for youth. Additionally, links have been added to the educational content in the STORIES and LEARN features to connect youth directly to the SHOP and support streamlined access to products. At the district and school level, pre-launch sensitisation meetings and launch events are planned with district officials, school administrators, parents, teachers, and school staff prior to implementation to increase stakeholder engagement with the platform, mitigate potential concerns about the content, and reduce gatekeeping. Teachers and students will attend a 1.5–2.5-day training, which includes all aspects of the platform and content and provides an opportunity to ask questions and gain confidence in the platform. Training for pharmacists will be improved to ensure that all pharmacy staff are equipped with the necessary knowledge and tools to manage the online ordering process and provide youth-friendly care. Willingness and ability to pay for products will also be further explored in the impact evaluation.

The results from our pilot study are supported by a systematic review of family planning services for youth, which found that confidentiality is highly valued among adolescents and that specialised provider training is necessary to improve uptake of FP/RH services.[29] Other studies have shown that providing sexual health education is a critical first step in improving access[30]; however, studies have also demonstrated that providing information alone is insufficient for changing behaviour, which requires reducing behavioural and structural barriers in addition to informational barriers.[31] CyberRwanda was designed to provide information as well as address some of these barriers through providing digital self-care for youth that extends beyond self-directed FP/RH education and acknowledges the importance of destigmatising FP/RH, reducing provider bias, prioritising confidentiality, and increasing autonomy and self-efficacy over adolescent well-being and care seeking. The impact evaluation planned for early 2021 will help determine whether this self-care model is effective in achieving these goals and shifting behaviours among the adolescent population.[22]

There are several potential limitations to consider when interpreting our pilot findings. Although efforts were made to ensure privacy during the study and reduce participant anxiety, students may not have reported FP/RH knowledge, care utilisation, or sexual activity accurately due to stigma and shame associated with discussing FP/RH. We adapted our survey strategy during the pilot to reduce participant anxiety and improve the quality of data, which included handing the tablets to participants for direct, private entry of all sensitive survey questions. To ensure accurate reporting for the impact evaluation, the survey has been redesigned and will employ a diverse range of methods to capture sexual activity, contraceptive use, and childbearing. In addition to handing the tablets to participants for all sensitive survey questions, students will respond to sensitive questions directly without being probed by an enumerator, questions will be asked in numerous ways and in several places throughout the survey, and follow-up questions will be used to confirm responses. In addition, there were a small number of interviews conducted with each group of stakeholders, and it is possible that some perspectives on the CyberRwanda program were not captured, including from potentially out-of-school youth accessing CyberRwanda at youth centres. Lastly, the pilot was conducted in only three districts, thus participants were not representative of the adolescent youth population in all districts in Rwanda. Nevertheless, the pilot study achieved its primary goals to provide valuable feedback on a new model intended to meet the FP/RH needs of youth and to finalise the procedures for the impact evaluation.

Taken together, these findings provide preliminary indications that CyberRwanda, a comprehensive sexuality education and online ordering platform for adolescents developed using a rights-based framework, is able to address several key areas in the theory of planned behaviour, including knowledge and beliefs, perceived behavioural control, intention, and actual control. The impact evaluation, a three-arm non-inferiority cluster randomised controlled trial, initiated in February 2021, will allow us to further assess the effectiveness of the program over a 24-month period with a robust study design and assess causality.[22] A separate evaluation of the program will be conducted in youth centres to understand the effectiveness of the program among adolescents who may not be in school.

Conclusion

Digital self-care interventions have the potential to reduce important barriers to FP/RH care for adolescents who lack access to information and youth-friendly services. CyberRwanda is an example of a rights-based, digital self-care intervention designed in partnership with youth that works across multiple health system building blocks – information, service delivery, health workforce. It aims to go beyond facilitating access to health products and services, to support informed choice, agency, self-efficacy and address the stigma associated with seeking FP/RH services for adolescents. The results from this pilot study provide an initial signal that the CyberRwanda program is relevant to the needs and lives of youth in Rwanda and that there is interest and demand for further engagement. These findings also serve to inform ongoing iterations of the digital platform and implementation model using a human-centred design methodology. Additional planned research examining the impact of this program on a larger scale will provide further evidence of its effectiveness within the school-aged adolescent population in Rwanda after a longer implementation period.

Click here for additional data file.