A quasi-experimental evaluation of an HIV prevention programme by peer education in the Anglican Church of the Western Cape, South Africa.

INTRODUCTION: Religion is important in most African communities, but faith-based HIV prevention programmes are infrequent and very rarely evaluated.
OBJECTIVE: The aim of this study was to evaluate the effectiveness of a church-based peer education HIV prevention programme that focused on youth.
DESIGN: A quasi-experimental study design compared non-randomly chosen intervention and control groups.
SETTING: This study was conducted in the Cape Town Diocese of the Anglican Church of Southern Africa. PARTICIPANTS: The intervention group of 176 teenagers was selected from youth groups at 14 churches and the control group of 92 from youth groups at 17 churches. Intervention and control churches were chosen to be as similar as possible to decrease confounding. INTERVENTION: The intervention was a 20-session peer education programme (Fikelela: Agents of Change) aimed at changing risky sexual behaviour among youth (aged 12-19 years). Three workshops were also held with parents. PRIMARY AND SECONDARY OUTCOME MEASURES: The main outcome measures were changes in age of sexual debut, secondary abstinence, condom use and numbers of partners.
RESULTS: The programme was successful at increasing condom usage (condom use score 3.5 vs 2.1; p=0.02), OR 6.7 (95% CI 1.1 to 40.7), and postponing sexual debut (11.9% vs 21.4%; p=0.04) absolute difference 9.5%. There was no difference in secondary abstinence (14.6% vs 12.5%; p=0.25) or with the number of partners (mean 1.7 vs 1.4; p=0.67) and OR 2.2 (95% CI 0.7 to 7.4).
CONCLUSION: An initial exploratory quasi-experimental evaluation of the Agents of Change peer education programme in a church-based context found that the age of sexual debut and condom usage was significantly increased. The study demonstrated the potential of faith-based peer education among youth to make a contribution to HIV prevention in Africa. Further evaluation of the effectiveness of the programme is, however, required before widespread implementation can be recommended.

RCT Entities:   Population Interventions Outcomes

Introduction

Two thirds of all people living with HIV are found in Sub-Saharan Africa (22.4 million of 33.4 million), although <10% of the world's population live here.1 AIDS has caused immense human suffering on this continent, and it has impacted many areas of people's lives. Effective prevention methodologies are urgently needed.2

The few countries that have successfully decreased national HIV prevalence have achieved these gains mostly by prioritising behaviour change among youth.3 Most HIV prevention programmes are run in schools, but churches provide an alternative setting. Churches are growing rapidly in Africa and religion is a meaningful aspect of most people's lives, whereas in many parts of the developed world, the number of church goers is dropping.4 5 In South Africa, according to the 2001 census, the majority of South Africans (79.8%) identified themselves as being affiliated to Christian churches.6 The church has the potential to become a key organisation in prevention work because of its reach into all communities and its strong history of voluntary work. Churches have also been shown to be effective partners in other health initiatives.7 8 In terms of HIV prevention, faith communities express clearer norms against premarital sex than other groups such as the media or peers.9 It has been identified that churches are of key importance in combating HIV.10 It is also recognised that the potential of churches is underutilised, given the influence and reach of faith-based organisations in African societies.11 12

For this reason, the Agents of Change programme was developed by the Fikelela AIDS Project in the Cape Town Diocese of the Anglican Church as an HIV prevention programme. A previous study indicated that Anglican youth engaged in sexual behaviour to the same extent as the general population and were just as much at risk of HIV.13 The goal of the Agents of Change programme was to reduce the vulnerability of youth to HIV infection using peer education as a methodology. Peer educators were selected by their youth group and trained to run a 20-session life skills programme, with the support of older facilitators.

This research seeks to fill the gap in knowledge regarding the effectiveness of church-based peer education programmes for HIV prevention in Africa. There are a limited number of church-based HIV interventions reported from high-income countries, but it is remarkably difficult to access studies on religiosity and sexual activity from Africa and most of the literature refers to the scarcity of research in this field.14–18 In addition, there are few evaluations of peer education programmes in Africa, considering the magnitude of the problem and the number of such programmes.19–22 This study is the only evaluation of a peer education programme based in an African church context that was identified to date.

The aim of the research was to evaluate the effectiveness of the Agents of Change programme in terms of the sexual practices of participating youth and to make recommendations regarding its further implementation. The objectives of the study were to measure changes in primary and secondary abstinence, condom use and number of partners that could be attributed to the intervention.

Methods

Study design

The evaluation was designed as a quasi-experimental study that compared non-randomly chosen intervention and control groups. This article reports on the outcomes of the intervention, and a more qualitative evaluation of the processes involved is reported on elsewhere.23

Setting

The study population was Anglican youth within the Cape Town Diocese of South Africa. The Cape Town Diocese consisted of 52 congregations situated in communities throughout the Cape Town metropole with an estimated Sunday attendance of 10 000 people.

Participants

Individual participants could not be randomly selected since the programme works with the whole youth group. All churches that showed interest in the programme were invited to take part, and due to the need for this commitment, it was not possible to randomly select intervention churches. Thus, it was decided to design the overall evaluation as a quasi-experimental study with control churches that were chosen to be as similar to the intervention churches as possible. Although the intervention was conducted with the youth group, the goal of the programme was to impact individuals, and therefore, behaviour change was analysed on an individual basis.

The intervention group was defined as churches that were running the programme. Intervention churches without similar control churches were not included in the study. Seven of the 14 included intervention churches that were situated in communities with the highest HIV rates in the Western Cape.24 These are mostly areas of relative socioeconomic deprivation with high levels of crime and drug use.25

The control group was defined as churches that were not running the programme. In the Anglican Church, churches are clustered geographically into ‘archdeaconries’; therefore, 17 control churches were selected from the same archdeaconry and in close proximity to the intervention churches.

The sample size was based on the estimate that 800 youth attended youth groups throughout all the churches over a 1-year period. A required total sample size of 260 was needed to determine the difference between the groups with 5% precision and 95% CIs. A paired t test sample size calculation suggested that a sample size of 260 should have 90% power to detect a standardised effect size of 0.2. It was assumed that the youth groups were homogeneous, and analysis of the clusters confirmed that there were no statistically significant differences among them in terms of sexual activity at baseline. Youth were excluded from the study if they did not obtain parental consent, if they were outside of the 12–19-year-old age bracket and if they poorly completed their baseline questionnaire. Following the first year, insufficient matched questionnaires were received, and it was necessary to continue data collection for a second year. The actual sampling process is shown in figure 1.

Figure 1

Sampling process for the study.

A final sample of 268 was therefore obtained, although it was not divided equally between the groups and churches were not clusters in the usual sense of a designed experiment with clustering. The final intervention sample of 176 was divided between 14 churches with a range of respondents from 2 to 39 per church and an average cluster size of 12.5. The final control sample of 92 was divided between 17 churches with a range of respondents from 1 to 22 per cluster and an average cluster size of 5.4.

The intervention group with paired predata and postdata (n=176) did not differ significantly from the whole intervention group (n=492) at baseline in terms of either age or gender. The control group with paired predata and postdata (n=92) did not differ significantly from the whole control group (n=580) in terms of gender but had a significantly lower mean age (15.0 vs 15.7 years, p<0.001). Those who dropped out of the control group were therefore older than those who stayed in. This is most likely due to chance as the control sample was much smaller rather than a systematic loss to follow-up of older teenagers across both groups.

The intervention

The Agents of Change programme aimed to reduce young people's vulnerability to HIV and teenage pregnancy by assisting them to build healthier relationships, by creating positive peer pressure and by improving communication with their parents. It is a comprehensive prevention programme, encouraging a delay in sexual activity, but promoting condom use and a reduced number of partners for those who are already sexually active.

The programme is based on various theories of behaviour change. The theory of diffusion of innovation suggests that peer educators better represent the social structure of the target group since adolescents are more likely to follow the behaviour of role models if they perceive them to be similar to themselves.26 The methods used during the sessions were based on Freirian theories of education, where the participation and empowerment of the people who are affected by a problem is essential in order to enact change.27 Thus, the programme was based on participatory education through discussion and role plays and not traditional didactic methods.28–30

Another goal of the programme was to increase self-efficacy, the belief that you can change, which is crucial for change to take place. According to Bandura's social learning theory, an increase in self-confidence, and a belief that one can effectively use the skills needed for a particular action, will increase the likelihood that a young person will take that action.27 The programme also aimed to increase social capital, which has been shown to protect youth from risky behaviour, through socialisation between peers and relationship building with adult mentors.31 32

Peer educators between the ages of 15 and 18 years were elected from the youth group by the members, while facilitators were the usual youth group leaders. Facilitators therefore were generally older young adults. At a 3-day training camp, peer educators were trained by staff members from Fikelela AIDS Project in the four roles of peer educators: to be role models who contribute to positive peer pressure, to be educators, to refer youth with problems to appropriate resources and to advocate for social development in the community. Facilitators were trained in their mentorship role, to do stakeholder analysis, monitoring and evaluation. Their mentorship role was to support the peer educators in the planning of presentations and to function as a ‘guide on the side’. At the heart of peer education is the belief that people do not change through providing information; they change when those around them change.33

The peer education programme consisted of 20 life skills sessions, with each session lasting 90 min and run over a 12-month period. The sessions started by looking at values and relationships and then explored teenage pregnancy, HIV, sexual coercion, drugs and alcohol. The peer educators presented these sessions to the young people using an approach that promoted interaction and participation. The facilitators assisted with preparation of the sessions, logistics and group discipline. They were also responsible for monitoring and evaluation and sending reports to Fikelela regarding the sessions completed.

In addition to the sessions presented by peer educators to the youth, three sessions were held with parents. The principles of motivational interviewing were adapted as core skills for the parents to learn in these workshops. These were collaboration, evocation, respect for choice and control, empathic listening, sharing information carefully, building confidence, reducing resistance and developing discrepancy.34 35 The three 2-h sessions had the goal of improving communication between parents and children and helping parents to understand the challenges their children face around issues of risky behaviour.

Data collection

The main outcome measures were changes in sexual activity, condom use and numbers of partners. These outcomes were measured by means of a self-completed preintervention and postintervention questionnaire. This questionnaire was developed from questions used in three previous surveys.36–38 The final questionnaire was also used in a previous youth survey within the Anglican Church.13 The questionnaire also collected information on potential confounding factors such as age, gender, place of residence, race, household members and types of school or other prevention programmes that they had attended.

The preintervention questionnaire was administered to all control and intervention churches at baseline. The postintervention questionnaire was administered to intervention churches once the 20 sessions of the programme had been completed (9–11 months after initiating the programme). In control churches, the questionnaire was repeated after 11 months. The programme was run in 2008 and 2009 and data were collected before and after the intervention in both years. Data were collected by independent research assistants who were students at a local Bible college. Each participant filled in the questionnaire after the youth group or confirmation class and they were anonymous.

Data analysis

Matched before and after data were collected on 176 participants in the intervention group and 92 in the control group. This was more than the recommended sample size of 260 but was unequally split between the two groups. The difficulty in collecting matched questionnaires was because of the fluctuation in youth group attendance from week to week, together with poor performance by some of the research assistants. Data were analysed by the Centre for Statistical Consultation at Stellenbosch University using Statistica Version 8. In questions with ordinal responses, such as for condoms, the answers were scored from 1 to 4 (eg, 1= never used, 2= sometimes used, 3= frequently used and 4= always used) and a mean score was calculated. The means of before and after scores were compared in both the intervention and control groups to test for significance using repeated measures analysis of variance or Friedman non-parametric tests. Analysis of variance was used to compare continuous to categorical variables and non-parametric methods for comparing ordinal to categorical variables. Categorical data were analysed by comparing the probability of change in the intervention group with that in the control group and testing for any significant difference using McNemar's test.

Ethical considerations

Ethical approval to conduct the research was obtained from the Health Research Ethics Committee of the University of Stellenbosch (N06/08/170). Permission was obtained from the Archbishop of Cape Town, Archbishop Njongonkulu Ndungane and from the local priests in each church. Young people had to give their assent and also provide written informed consent from their parents or guardians. Confidentiality was preserved at all times, and questionnaires were filled in anonymously.

Results

Comparison of intervention and control groups at baseline

The intervention and control groups are compared at baseline in table 1. The mean age of the intervention group was 15.3 years and the control group was 15.0 years (p=0.22). Just under two thirds of the youth were female and almost all were living in urban communities on the Cape Flats. The majority of youth would regard themselves as belonging to the coloured community. Just under a third were living in single parent families and only a small minority with other relatives or friends. Almost all attended public sector schools, although government model ‘C’ schools were historically white schools with additional income from fees and tend to be better resourced. Very few attended private schools in keeping with the poor socioeconomic circumstances of these communities. Most were exposed to sexual education at school as part of compulsory life orientation classes and just under a third had been exposed to other peer education programmes. Not surprisingly the majority of youth were Christian, although as youth groups were open to the whole community, a few youth from other religions were also included.

Table 1

Comparison of control with intervention groups at baseline

Variable	Intervention, N=176, n (%)	Control, N=92, n (%)	p Value
Gender
Male	63 (35.8)	34 (37.0)	0.85
Female	113 (64.2)	58 (63.0)
Place of residence
Urban	149 (87.6)	85 (94.4)	0.08
Rural	21 (12.4)	5 (5.6)
Race
Coloured	165 (93.8)	92 (100.0)	0.05
White	5 (2.8)	0 (0.0)
Black	6 (3.4)	0 (0.0)
Live with
One parent	52 (29.5)	28 (30.8)	0.84
Both parents	115 (65.3)	59 (64.8)	0.93
Another relative	7 (4.0)	49 (4.4)	0.87
Friend	2 (1.1)	0 (0)	0.31
School
Local government	122 (71.8)	55 (61.1)	0.05
Model ‘C’	41 (24.0)	31 (34.4)	0.09
Private school	1 (0.6)	2 (2.2)	0.87
Higher education institution	4 (2.3)	2 (2.2)	0.31
Prevention programmes at school
Peer education	69 (31.3)	30 (27.0)	0.29
Life orientation class by teacher	153 (68.9)	81 (73.0)	0.69
Religion
Christian	169 (97.7)	91 (98.9)	0.35
Other	4 (2.3)	1 (1.1)	0.49

There were no significant differences between the groups in the socio-demographic variables measured at baseline apart from race and attendance at local government school where there was a borderline significant difference (p=0.05). None of these potential confounding factors were considered sufficiently different to warrant adjusting for in the analysis of the final outcomes.

There were no significant differences between the intervention and control groups at baseline in terms of sexual activity (17.6% vs 10.3%, p=0.12), condom use (mean score 2.2 vs 2.2, p=0.97) or number of partners (mean number 1.8 vs 1.6, p=0.52).

Effect of the intervention on sexual intercourse

Table 2 shows the effect of the intervention on sexual intercourse. Primary abstinence refers to those that continued to defer their sexual debut, while secondary abstinence refers to those who were sexually active and stopped having a sexual partner during the year. The number of youth who started having sex during the year was significantly less in the intervention group indicating a shift in the age of sexual debut. In the intervention group, there were 134 not yet sexually active youth, of whom 16 became sexual active during the course of the year. In the control group, there were 84 who were not yet sexually active and 18 of them became sexually active. The absolute difference between the groups was a 9.5% increase in those delaying their sexual debut. There was no difference between the groups in terms of secondary abstinence.

Table 2

Effect of intervention on abstinence

Variable	Intervention, n=175, n (%)	Control, n=92, n (%)	p Value
Primary abstinence	118/134 (88.1)	66/84 (78.6)	0.04
Secondary abstinence	6/41 (14.6)	1/8 (12.5)	0.25

Effect of the intervention on number of sexual partners and condom use

Table 3 shows that there was no effect on the mean number of sexual partners, but there was a significant increase in the use of condoms among the intervention group. The OR of the intervention for increased condom use was 6.7 (95% CI 1.1 to 40.7) and for decreased number of partners was 2.2 (95% CI 0.7 to 7.4).

Table 3

Effect of intervention on number of sexual partners and condom use

Variable	Intervention (n=33)		Control (n=7)		p Value
	Pre-mean (95% CI)	Post-mean (95% CI)	Pre-mean (95% CI)	Post-mean (95% CI)
No. of partners	1.7 (1.4 to 2.1)	1.7 (1.3 to 2.0)	1.7 (0.9 to 2.5)	1.4 (0.6 to 2.2)	0.67
Condom use	2.2 (1.7 to 2.6)	3.5 (3.2 to 3.8)	2.3 (1.7 to 3.0)	2.1 (1.5 to 2.9)	0.02

Discussion

Key findings

The findings of this quasi-experimental study suggest that church-based peer education can raise the age of sexual debut for young people and increase the use of condoms among those who are sexually active. There was no effect on the numbers of sexual partners or on secondary abstinence.

Comparison with other studies

A meta-analysis of HIV youth prevention programmes in Sub-Saharan Africa reported that abstinence was increased in 6/17 (35%) studies, the number of partners was reduced in 10/15 (67%) studies and condom use was increased in 14/20 (70%) studies.16 29 39 All the programmes, except for one, made an impact in at least one area, seven in two areas and only one in all three areas. Although a reduction in the number of partners was found in two thirds of these other programmes, this was not shown in the Agents of Change. The potential range of effects of the Agents of Change programme, therefore, appears broadly similar in scope to these other programmes.

Unlike some faith-based organisations, the Agents of Change programme used a comprehensive rather than an abstinence-only approach. The use of condoms as part of a comprehensive programme is still a controversial issue for many churches, particularly the Roman Catholic and Pentecostal denominations. Abstinence-only programmes, however, are not effective in high-income countries and may even lead to stigmatisation.40–42

In terms of process, one of the reasons for the effect of the programme may be the length, as the programme took place over a year, which enabled the peer educators and facilitators to have a more persistent influence on the youth. Other shorter programmes appear to have less effect on sexual risk factors.19 The positive findings of this study suggest that the theoretical basis for the intervention was appropriate for encouraging sexual behaviour change among youth. A qualitative process evaluation of the programme, which may help determine which aspects of the design were most useful, was also conducted and is reported on elsewhere.23

Limitations

The findings of this study are limited by the possibility of bias and confounding from the quasi-experimental study design that did not randomly select churches or individual participants. The results therefore should be interpreted with caution in terms of inferring cause and effect as opposed to an association. A larger pragmatic clustered randomised controlled trial in future would give stronger evidence for the effectiveness of the programme.

The churches were self-selecting since the clergy who did not think that these sorts of issues should be addressed in the church would not send their young people to take part in the Agents of Change programme. It was not possible to randomly select all the study sites from only willing churches as all these were needed to generate the intervention group sample size. This may mean that the selected churches were more open to discussion of sexual behaviour and could limit the generalisability of the results to the broader church. Control churches were chosen from the same area and comparison of the control and intervention groups did not show any important differences in the potential confounding factors that were measured. Television viewing was not included as a confounding factor.

As the youth group clusters were homogenous at baseline in terms of their sexual activity, the subsequent analyses done were applicable and the sample size calculation adequately powered. However, it was not possible to adjust the outcomes for the possible effect of clustering due to the variability in size of the clusters. There was also insufficient power to analyse the results in terms of the clusters themselves, and analysis was performed at the level of the individual.

A weakness of the study was the difficulty in obtaining sufficient numbers of paired before and after questionnaires, particularly in the control group, which led to a reduction in the power of the study. Although reasonable numbers were achieved at baseline, the follow-up questionnaires were far fewer. This is because, unlike in the school setting, young people do not attend youth group every week, and so, there were different participants at different times. This was particularly problematic when it came to sub-analysis such as for just the sexually active, males versus females or oral and anal sex.

There were two possible causes of reporting bias. In the first place, the study was based on self-reported behaviour in a church context, and there is therefore the possibility for reporting bias, particularly under-reporting of sexual behaviour. However, the same type of data collection on sexual behaviour was used in a previous study, when the results from Anglican youth were similar to Western Cape statistics obtained in school surveys.13 Self-reporting is the usual method for measuring sexual behaviour in surveys.23 43

The second possible cause of reporting bias was that in the intervention churches, the young people were coming into contact with peer educators or facilitators that they liked and wanted to impress, so there might have been a danger of under-reporting. This social desirability bias is a recognised problem, but the data from the questionnaires were triangulated with information from the monitoring process to detect any obvious discrepancies between qualitative and quantitative findings.23 It was also collected anonymously, so that neither the facilitators nor peer educators knew what was written.

Implications and recommendations

The WHO conducted a systematic review of HIV interventions in the developing world in order to ascertain the types of intervention that should be rolled out.44 Interventions that worked through an existing organisation were judged to be more feasible, to have less potential for adverse outcomes, to be more readily acceptable and more likely to have a moderate effect size and other social benefits. They concluded that only a moderate threshold of evidence was required before recommending implementation of these types of programmes.

This framework has been used to help interpret the findings of this study and their implications for further implementation and evaluation. In addition to the quasi-experimental study reported here, a process evaluation was also conducted using Outcome Mapping as a methodology.23 This process evaluation also supports the view that the Agents of Change programme demonstrated all the potential strengths of existing organisations that were identified by the WHO44:

Feasibility: it was feasible to run the programme through the existing church structure and youth groups.

Lack of potential for adverse outcomes: no adverse outcomes were detected in the process evaluation of the study.

Acceptability: the programme was acceptable to the church leadership and the youth, particularly in the more at-risk communities.

Potential coverage: the Anglican Church constitutes 4.8% of the South African population6 and interventions with even a moderate effect could impact a large number of people. The intervention may also be suitable for other denominations.

Other health or social benefits: youth involved in the programme also become involved in the youth group at church and other church-based activities, such as community outreach and social events. The intervention may also have benefits on other lifestyle-related risks such as teenage pregnancy and substance abuse, although these were not measured in this study.

Despite the limitations of the study design and the relatively modest effect, the Anglican Church has decided to continue implementing the Agents of Change programme. Widespread implementation, however, should await further evaluation of the effectiveness of the programme. This study should be considered as an initial exploratory study with positive findings that require conformation in a larger experimental study.

Unanswered questions and future research

Ideally, such ongoing evaluation should involve a randomised controlled trial among interested churches that recruits and follows up successfully a much larger sample size with the possibility of subgroup analysis. Future research should also be conducted into the adaptation of the programme for different socioeconomic and cultural settings as well as other churches with differing theologies and practice regarding the use of condoms. Future research should also attempt to explore some of the other potential social and health benefits of the programme.

Interventions run by peers and those run by adults both appear to have moderate effects on youth sexual behaviour.45 It seems in general terms that those with adults only are more effective at presenting the abstinence option and those with peers are more successful with condom promotion. Peer education programmes, therefore, can be effective, but it is not clear whether these programmes are effective because they are run by peer educators or whether they might be more effective if the same programmes were run by adults. This is a question worth pursuing in future research.

Conclusions

In conclusion, an initial exploratory quasi-experimental evaluation of the Agents of Change peer education programme in a church-based context found that the age of sexual debut and condom usage was significantly increased. The study demonstrated the potential of faith-based peer education among youth to make a contribution to HIV prevention in Africa. Further evaluation of the effectiveness of the programme is, however, required before widespread implementation can be recommended.