Health outcomes of youth development programme in England: prospective matched comparison study.

OBJECTIVE: To evaluate the effectiveness of youth development in reducing teenage pregnancy, substance use, and other outcomes.
DESIGN: Prospective matched comparison study.
SETTING: 54 youth service sites in England. PARTICIPANTS: Young people (n=2724) aged 13-15 years at baseline deemed by professionals as at risk of teenage pregnancy, substance misuse, or school exclusion or to be vulnerable. INTERVENTION: Intensive, multicomponent youth development programme including sex and drugs education (Young People's Development Programme) versus standard youth provision. MAIN OUTCOME MEASURES: Various, including pregnancy, weekly cannabis use, and monthly drunkenness at 18 months.
RESULTS: Young women in the intervention group more commonly reported pregnancy than did those in the comparison group (16% v 6%; adjusted odds ratio 3.55, 95% confidence interval 1.32 to 9.50). Young women in the intervention group also more commonly reported early heterosexual experience (58% v 33%; adjusted odds ratio 2.53, 1.09 to 5.92) and expectation of teenage parenthood (34% v 24%; 1.61, 1.07 to 2.43).
CONCLUSIONS: No evidence was found that the intervention was effective in delaying heterosexual experience or reducing pregnancies, drunkenness, or cannabis use. Some results suggested an adverse effect. Although methodological limitations may at least partly explain these findings, any further implementation of such interventions in the UK should be only within randomised trials.

Introduction

Youth development programmes aim to promote overall personal development, self esteem, positive career and other aspirations, and good relationships with adults among vulnerable young people, in order to promote motivation to avoid teenage pregnancy and other negative health and social outcomes.1 Alongside education on sex and drugs, these programmes offer activities such as education, social skills development, mentoring, arts, sports, and volunteering. The most notable of these, the Children’s Aid Society’s Carrera programme was an intensive after school intervention combining youth development, sexuality education, and regular sexual health clinic check-ups. This was reported as delaying young women’s sexual experience, increasing their use of contraception, and reducing pregnancies when implemented in New York, but no such benefits were seen for young men.2 Several influential reviews subsequently identified youth development as a promising approach to reducing teenage pregnancies.3 4 However, studies of attempted replications of the Carrera programme in the United States did not report benefits for young women or men, and recent reviews have called for further evaluation.1 5 6 We report on sexual health and other outcomes of a youth development intervention implemented in England. We also provide some key findings on process, although this will be covered further in a forthcoming paper.

The Carrera programme included tutoring as well as work preparation, sex and drugs education, arts and sports, and referrals to health interventions. Informed by the Carrera programme and other youth development programmes, the Young People’s Development Programme (YPDP) was a three year (April 2004-March 2007) initiative funded by England’s Department of Health and targeting young people aged 13-15 at entry deemed by teachers or other care professionals to be at risk of teenage conception, substance misuse, or exclusion from school. These participants were recruited in three annual cohorts. The programme aimed to reduce teenage pregnancy, substance use, and other outcomes (table 1) through an intensive programme focused on overall personal development. Content was to include education, training/employment opportunities, life skills, mentoring, volunteering, health education (particularly sexual health and substance misuse), arts, sports, and advice on accessing services (such as family planning and substance misuse services).

Table 1

 Aims and outcomes of Young People’s Development Programme

Aim	Outcome at follow-up 1	Outcome at follow-up 2
Reduced conceptions; reduced sexually transmitted infections	Heterosexual sex; two or more sexual partners in previous 6 months; condom use less than half the time in previous 6 months; difficulty suggesting condom use to partner	Heterosexual sex; two or more sexual partners in previous 3 months; condom use less than half the time in previous 6 months; post-baseline pregnancy (young women) and causing pregnancy (young men)
Reduced illegal drug use	Weekly or more frequent cannabis use in previous 6 months	Weekly or more frequent cannabis use in previous 3 months
Reduced alcohol consumption	Monthly or more frequent drunkenness in previous 6 months	Monthly or more frequent drunkenness in previous 3 months
Improved mental health and self esteem	Often worried about things in previous few weeks; often angry/lost temper in previous few weeks; ease discussing personal things with close friend	Often worried about things in previous few weeks; low self esteem (adapted Rosenberg scale); ease discussing personal things with close friend
Reduced school exclusions	Temporary school exclusion in previous 6 months	Temporary school exclusion in previous 3 months
Reduced school absenteeism	Truancy in previous 6 months	Truancy in previous 3 months
Improved educational attainment	Dislike of school	None
Increased post-16 participation in education, employment, and training; increased preparedness/aspirations for adult life	Expects to be parent by age 20; does not expect to be in steady job by age 20	Currently in education, employment, or training
Reduced offending/convictions	Contact with police (stopped, told off, picked up) in previous 6 months	Contact with police (stopped, told off, picked up) in previous 6 months; official warnings or convictions in previous 6 months

Measures generally focused on standard three month periods at follow-up 2 but six month periods at follow-up 1 to allow more sensitivity to risk behaviours among younger participants.

Through competitive tendering, the Department of Health identified 27 existing projects to deliver the YPDP, which received additional funding and support from the National Youth Agency, a non-governmental agency. Tenders were judged on the quality of the proposed work, local deprivation, and teenage pregnancy rates and to ensure geographical spread and ethnic/gender diversity of participants. Although the YPDP was influenced by the Carrera programme, it intentionally differed from the outset in that provision was less tightly defined and young people were targeted on their perceived behavioural risk and were to be involved for 6-10 hours a week for one year, compared with up to 15 hours a week for three years in the Carrera programme.

Methods

The Department of Health commissioned us to independently evaluate the process, outcomes, and costs of the YPDP. A trial randomising individual participants was not feasible because groups of young people were to be referred to the YPDP together. A cluster randomised trial, whereby agencies would be randomly allocated to provide the YPDP or continue normal work, was ruled out by the competitive tendering outlined above. We therefore opted for a prospective matched cluster comparison with pre-intervention/post-intervention data from the young people in 27 intervention sites and 27 comparison sites matched by region, local deprivation, teenage pregnancy rates, area (urban, rural, or seaside) and sector (voluntary or statutory). In comparison sites, we recruited from youth service providers that had bid and were shortlisted for, but did not receive, YPDP funds, as well as pupil referral units providing education to young people not attending mainstream schools (which in YPDP sites were referrers to the programme). Youth service providers in comparison sites also worked in deprived areas with high rates of teenage pregnancy and had failed to be selected for YPDP primarily on the basis of quality of tender. We recruited young people at these sites by asking workers to identify young people aged 13-15 who were at risk of teenage pregnancy, substance misuse, or exclusion from school (that is, as for the YPDP), although in practice field workers sometimes asked workers to identify “vulnerable” young people. We thus aimed to recruit young people in comparison sites who might have been referred to the YPDP had it been delivered in their area.

As required by the funders, we aimed to examine the effects of YPDP on multiple outcomes to reflect the broad aims of the programme and pre-hypothesised measures before analysis (table 1). We collected self reported questionnaire data from young people in the YPDP and comparison groups at baseline (shortly after joining) and approximately nine and then 18 months later. Our original statistical power calculation was based on 35 YPDP sites (as was originally planned) and 35 comparison sites, with an estimated 2300 young people participating in each arm. This sample size, with a 5% level of significance, 80% power, and an estimated intracluster correlation coefficient of 0.02, would enable us to detect a 50% change in teenage pregnancies on the basis of an assumed 4.5% teenage pregnancy rate informed by a previous trial.7 We based our sample size calculations on teenage pregnancy, as other outcomes were likely to be more prevalent. With the subsequent reduction of YPDP sites to 27 by the Department of Health (and therefore similarly the number of comparison sites) and lower than expected average numbers of young people at each site (around 40) the study was powered to detect a reduction in pregnancy rates of 62% at 5% significance.

Young people completed questionnaires at the programme site or a nearby site (for example, a minibus) or by telephone interview. In telephone interviews or for participants with literacy problems, a researcher read out questions and indicated response options. Of the 2371 young people who participated in the YPDP, 1637 (69%) completed baseline questionnaires, as did 1087 young people in comparison sites (table 2). Follow-up 1 questionnaires were completed by 1054 (64%) in YPDP sites and 599 (55%) in comparison sites. Timing of the evaluation meant that follow-up 2 could be completed only by young people recruited in the first two (2004/5 and 2005/6) recruitment cohorts. In YPDP sites, 566 (43% of the baseline pool) completed follow-up 2 questionnaires; 338 (39%) of comparison participants did so. The vast majority of those who did not complete questionnaires were missed because of irregular attendance (baseline, follow-up 1) or because contact details changed and they could not be located (follow-up 2). Monitoring data indicated that YPDP participants who did not complete baseline questionnaires spent less time on the YPDP (mean 89 v 225 hours), were the same age at entry (mean 14.2 years), and were significantly more likely (P<0.05) to be of black/minority ethnicity (25% v 22%) or female (38% v 37%). Participants gave informed, signed consent to data collection, and data were stored in anonymised form.

Table 2

 Response rates and baseline characteristics. Values are numbers (percentages) unless stated otherwise

Survey wave/baseline measure	Intervention group	Comparison group
Participated in YPDP:
Overall	2371	NA
Years 1 and 2 only	1864	NA
Completed baseline:
Overall	1637 (69% of participants)	1087
Years 1 and 2 only	1333 (72% of participants)
Completed first follow-up	1054 (64% of baseline pool; 44% of participants)	599 (55% of baseline pool)
Completed second follow-up	566 (43% of years 1 and 2 baseline pool; 30% of years 1 and 2 participants)	338 (31% of baseline pool)
Male sex**	1015/1637 (62)	610/1086(56)
Mean age (years)	14.56	14.65
Non-private housing **	1095/1505 (73)	613/1006 (61)
Lone parent family**	627/1337 (47)	391/945 (41)
Workless household*	493/1260 (39)	319/917 (35)
Language other than English mainly spoken at home	97/1631 (6)	47/1076 (4)
Black or minority ethnicity*	360/1586 (23)	184/1029 (20)
Drunk monthly or more often in previous 6 months *	482/1352 (36)	388/964 (40)
Drug use in previous 6 months	1179/1379 (86)	846/987 (86)
Experience of heterosexual sex**	562/1637 (34)	438/1087 (40)
No contraceptive use at first sex	115/583 (20)	101/450 (22)
No contraceptive use at latest sex	126/562 (22)	89/438 (20)
Dislikes school*	913/1365 (67)	608/974 (62)
Low parental interest in school achievement	76/1362 (6)	54/974 (6)
Truanting in previous 6 months **	584/1272 (46)	356/892 (40)
Temporary school exclusion in previous 6 months**	550/1262 (44)	330/873 (38)
Permanent exclusion from school**	183/1207 (15)	210/873 (24)
Attendance at pupil referral unit (at first follow-up)**	113/1043 (11)	119/595 (20)
Contact with police in previous 6 months	715/1354 (53)	522/976 (53)
Most friends had contact with police in previous 6 months	379/1353 (28)	291/964 (30)
Difficult to discuss problems with mother/female guardian	383/1330 (29)	268/960 (28)
Difficult to discuss problems with father/male guardian	501/1132 (44)	393/856 (46)
Does not expect be in steady job by age 20	144/1357 (11)	91/969 (9)
Expects to be parent by age 20	646/1342 (48)	474/946 (50)
Expects to be at college/university by age 20	418/1346 (31)	328/946 (35)

NA=not applicable; YPDP=Young People’s Development Programme.

*P<0.05 **P<0.01.

We examined outcomes by logistic regression, reporting odds ratios, both unadjusted and adjusted for age and measures of pre-hypothesised potential confounders that differed significantly between arms at baseline (table 2). We accounted for the clustering of data in all analyses and excluded participants from analyses if relevant data were missing. We present results for pregnancy stratified by gender, but for other outcomes we present overall results, only stratifying by gender if interactions were significant.

To check whether differential attrition at the follow-ups might explain our findings, we also did a weighted analysis. We identified baseline variables that predicted whether participants provided data at each follow-up (available on request) and created inverse probability weights so that outcomes reported by participants who did report data at that follow-up but had a low probability of doing so would be given more weight (www.lshtm.ac.uk/msu/missingdata/weighting_web/index.html). As an additional check, we did an analysis of follow-up 2 outcomes by using propensity scores to balance covariates in the two groups. We included a categorical propensity score covariate in models that also accounted for the clustered nature of the data.8 We computed the propensity score by using logistic regression with the dependent variable being receipt of the intervention and the independent variables being the baseline exposures listed in table 2. We also did a small number of sensitivity analyses to examine whether the way the programme was implemented had any effect. These analyses compared participants who attended agencies where participation was voluntary versus those where participation was required by the school, attended agencies judged by the National Youth Agency to deliver high versus moderate versus adequate quality work, participated for more versus fewer overall hours, and experienced a more versus less holistic package of participation (all versus comparison group). Our methods for evaluating these processes are reported elsewhere.9

Results

Some overall baseline differences existed (table 2). For some (such as housing, family structure) YPDP participants were more vulnerable, and for others (such as alcohol consumption, heterosexual experience) comparison participants were more vulnerable. We also checked for baseline differences by gender, and in general the results were consistent with the differences found overall. Overall, 60/622 young women in the YPDP group reported using no contraception at most recent sex compared with 33/477 comparison participants. Overall, 66/1015 young men in the YPDP group reported using no contraception at first sex compared with 65/610 in the comparison group. These differences were statistically significant (P<0.05) in analyses that included only those with baseline heterosexual experience but not analyses including, respectively, all young women and all young men at baseline.

Tables 3 and 4 show the outcomes at the two follow-ups. Regarding health outcomes, significantly more pregnancies were reported post-baseline among young women in the YPDP group (38) than in the comparison group (13); this association remained significant after adjustment for all pre-hypothesised confounders (table 4) and also after confirmatory adjustment for no contraception at most recent sex at baseline. The difference was greater among the first cohort but remained among subsequent cohorts. Pregnancies were reported in 16 YPDP sites (median 1 per site, interquartile range 0-2). We found no significant differences in the proportion of young men who reported awareness of causing a pregnancy. Although we found no significant differences between young people in the YPDP and comparison groups overall in those expecting to be a parent by age 20, female YPDP participants were significantly more likely than comparisons to report this expectation at follow-up 1; this remained significant in all models.

Table 3

 Outcomes at follow-up 1. Values are numbers (percentages) unless stated otherwise

Outcomes	Intervention group	Comparison group	Unadjusted odds ratio* (95% CI); P value	Adjusted odds ratio† (95% CI); P value	Weighted‡ adjusted odds ratio† (95% CI); P value
Heterosexual sex before age 16	435/999 (44)	236/578 (41)	1.12 (0.66 to 1.90); 0.68	1.37 (0.85 to 2.20); 0.19	1.28 (0.78 to 2.11); 0.31
Of those with heterosexual experience:
≥2 sexual partners in previous 6 months	162/474 (34)	92/261 (35)	0.96 (0.65 to 1.41); 0.83	1.20 (0.71 to 2.04); 0.48	1.28 (0.76, to, 2.15); 0.34
Condom use in less than half sexual encounters in previous 6 months	121/447 (27)	65/243 (27)	1.01 (0.72 to 1.43); 0.93	1.11 (0.67 to 1.83); 0.67	1.29 (0.76 to 2.20); 0.34
Perceived difficulty in initiating condom use	115/966 (12)	61/544 (11)	1.07 (0.77 to 1.48); 0.68	1.05 (0.75 to 1.48); 0.75	1.07 (0.75 to 1.51); 0.71
Cannabis use weekly or more in previous 6 months	156/1036 (15)	74/592	1.24 (0.66 to 2.34); 0.50	1.49 (0.79 to 2.82); 0.22	1.41 (0.75 to 2.68); 0.28
Drunkenness monthly or more in previous 6 months	388/1042 (37)	218/594 (37)	1.02 (0.64 to 1.64); 0.92	1.02 (0.67 to 1.56); 0.92	0.98 (0.65 to 1.47); 0.93
Dislike school	673/1027 (66)	363/590 (62)	1.19 (0.70 to 2.02); 0.52	1.09 (0.72 to 1.67); 0.66	1.08 (0.71 to 1.64); 0.70
Truancy in previous 6 months	341/730 (47)	119/409 (29)	2.14 (1.19 to 3.84); 0.01	2.13 (1.22 to 3.72); <0.01	2.16 (1.23 to 3.77); 0.01
Temporary school exclusion in previous 6 months	265/741 (35)	99/414 (24)	1.77 (0.98 to 3.19); 0.06	1.67 (0.91 to 3.07); 0.09	1.60 (0.83 to 3.07); 0.16
Contact with police	466/1037 (45)	245/591 (42)	1. 15 (0.72 to 1.84); 0.55	1.25 (0.82 to 1.91); 0.29	1.17 (0.78 to 1.75); 0.43
Worry often in previous few weeks	624/1047 (60)	360/598 (60)	0.98 (0.71 to 1.34); 0.88	0.87 (0.60 to 1.27); 0.47	0.82 (0.56 to 1.21); 0.32
Often angry in previous few weeks	472/1048 (45)	275/599 (46)	0.97 (0.73 to 1.27); 0.80	0.89 (0.64 to 1.22); 0.44	0.94 (0.69 to 1.25); 0.66
Difficulty discussing personal things with close friend	154/990 (16)	82/563 (14)	1.08 (0.74 to 1.59); 0.69	1.29 (0.82 to 2.03); 0.26	1.40 (0.90 to 2.18); 0.13
Expects to be parent by age 20	339/1010 (34)	176/579 (30)	1.16 (0.80 to 1.68); 0.44	1.22 (0.89 to 1.67); 0.22	1.18 (0.84 to 1.64); 0.34
Expects to be in steady job by age 20	100/1034 (10)	47/581 (8)	1.22 (0.72 to 2.06); 0.46	0.91 (0.54 to 1.23); 0.72	0.98 (0.59 to 1.61); 0.92

*Adjusted for cluster only.

†Adjusted for baseline housing type, workless household, lone parent household, ethnicity, dislike of school, truancy, temporary and permanent school exclusion, heterosexual experience, alcohol use, age, and gender, plus pupil referral unit attendance at follow-up 1.

‡Analysis weighted for missing returns at this follow-up and adjusted for above factors.

Table 4

 Outcomes at follow-up 2. Values are numbers (percentages) unless stated otherwise

Outcomes	Intervention group	Comparison group	Unadjusted odds ratio* (95% CI); P value	Adjusted odds ratio† (95% CI); P value	Weighted‡ adjusted odds ratio† (95% CI); P value
Heterosexual sex before age 16	294/537 (55)	130/326 (40)	1.82 (0.97 to 3.42); 0.06	1.51 (0.84 to 2.72); 0.16	1.79 (0.96 to 3.36); 0.07
Of those with heterosexual experience:
≥2 sexual partners in previous 3 months	101/343 (29)	42/166 (25)	1.23 (0.70 to 2.16); 0.46	1.03 (0.54 to 1.98); 0.91	1.16 (0.57 to 2.40); 0.67
Used condoms less than half sexual encounters in previous 3 months	69/276 (25)	38/149 (26)	0.97 (62 to 1.53); 0.91	0.99 (0.65 to 1.51); 0.95	1.03 (0.66 to 1.61); 0.90
Cannabis used weekly or more in previous 3 months	88/556 (16)	38/338 (11)	1.48 (0.71 to 3.11); 0.29	1.44 (0.72 to 2.89); 0.30	1.97 (0.93 to 4.17); 0.08
Drunkenness monthly or more in previous 3 months	216/557 (39)	109/337 (32)	1.32 (0.75 to 2.35); 0.39	1.07 (0.68 to 1.70); 0.76	1.20 (0.78 to 1.84); 0.39
Truancy in previous 3 months	58/270 (21)	24/166 (15)	1.62 (0.66 to 3.95); 0.28	1.53 (0.64 to 3.66); 0.33	1.82 (0.69 to 4.81); 0.22
Temporary exclusion from school in previous 3 months	43/320 (13)	21/178 (12)	1.16 (0.50 to 2.70); 0.73	1.00 (0.35 to 2.85); 0.99	1.09 (0.32 to 3.78); 0.88
Currently not in education, employment, or training	63/566 (11)	35/338 (10)	1.08 (0.52 to 2.24); 0.82	0.68 (0.34 to 1.36); 0.28	0.62 (0.29 to 1.34); 0.22
Contact with police in previous 6 months	196/553 (35)	105/332 (32)	1.19 (0.69 to 2.03); 0.52	0.87 (0.46 to 1.62); 0.65	0.90 (0.48 to 1.69); 0.75
Official warnings or convictions in previous 6 months	140/556 (25)	61/335 (18)	1.51 (0.83 to 2.76); 0.17	1.43 (0.82 to 2.51); 0.20	1.67 (0.90 to 3.13); 0.10
Low self esteem (adapted Rosenberg scale)	144/557 (26)	92/337 (27)	0.93 (0.57 to 1.52); 0.76	0.99 (0.63 to 1.56); 0.97	1.02 (0.62 to 1.67); 0.94
Often worry in previous few weeks	396/556 (71)	251/337 (74)	0.85 (0.62 to 1.15); 0.29	1.01 (0.76 to 1.36); 0.90	1.09 (0.81 to 1.47); 0.56
Difficulty in discussing personal things with close friend	59/557 (11)	43/338 (13)	0.81 (0.42 to 1.58); 0.53	0.80 (0.43 to 1.49); 0.48	0.75 (0.35 to 1.60); 0.45
Teenage pregnancy since baseline (young women)§	38/244 (16)	13/202 (6)	2.68 (1.35 to 5.32); <0.01	3.55 (1.32 to 9.50); 0.01¶	5.48 (2.18 to 13.75); <0.01¶
Caused a teenage conception since baseline (young men)	37/348 (11)	17/160 (11)	1.00 (0.44 to 2.28); 0.99	0.86 (0.35 to 2.07); 0.72	1.18 (0.45 to 3.09); 0.73

*Adjusted for cluster only.

†Adjusted for cluster, age at follow-up, gender, pupil referral unit attendance, and baseline variables: tenure, dislike of school, truancy, temporary and permanent exclusions from school, having had heterosexual sex, frequency of drunkenness, ethnicity, and living in lone parent household or one where no adults were working.

‡Analysis weighted for missing returns at this follow-up and adjusted for above factors.

§Includes pregnancies reported at follow-up 2 plus those reported by young women at follow-up 1 who did not complete follow-up 2.

¶In addition to adjustments mentioned above, this model also adjusts for pregnancy pre-baseline and ward level rates of teenage pregnancy (derived from young woman’s postcode area).

At first and second follow-up, non-significantly more young people in the YPDP group than in the comparison group reported heterosexual sex (tables 3 and 4) but with a significant gender interaction: significantly more young women in the YPDP group than in the comparison group reported heterosexual sex at follow-up 2 (table 5); this remained in all models. We found no differences in the proportion of young people who reported two or more partners or regular use of condoms at follow-up 1 or 2. No significant differences existed between YPDP participants and comparison participants in any measure of substance use or in worry, anger, low self esteem, or difficulty talking about personal matters to friends.

Table 5

 Outcomes at follow-up 1 and 2 stratified by sex. Values are numbers (percentages) unless stated otherwise

Outcomes	Young women			Young men			Unadjusted odds ratio* (95% CI); P value			Adjusted odds ratio† (95% CI); P value				Weighted‡ adjusted odds ratio† (95% CI); P value
	Inter-vention	Compa-rison		Inter-vention	Compa-rison		Young women	Young men		Young women	Young men			Young women	Young men
First follow-up
Temporary school exclusion in previous 6 months	97/294 (33)	31/211 (15)		168/447 (38)	68/203 (34)		2.86 (1.28 to 6.39); 0.01	1.20 (0.72 to 1.98); 0.48		3.65 (1.16 to 11.43); 0.03		1.10 (0.59 to 2.04); 0.76		3.38 (1.14 to 10.1); 0.03	1.07 (0.55 to 2.12); 0.83
Often angry in previous few weeks	222/404 (55)	136/285 (48)		250/644 (39)	139/314 (44)		1.34 (0.93 to 1.93); 0.12	0.80 (0.59 to 1.08); 0.14		1.05 (0.59 to 1.85); 0.87		0.76 (0.52 to 1.11); 0.16		1.17 (0.67 to 2.03); 0.57	0.81 (0.56 to 1.17); 0.25
Expects to be/already parent by age 20	131/390 (34)	66/278 (24)		208/620 (34)	110/301 (37)		1.62 (1.04 to 2.54); 0.03	0.88 (0.57 to 1.34); 0.53		1.61 (1.07 to 2.43); 0.02		0.95 (0.60 to 1.52); 0.84		1.61 (1.07 to 2.41); 0.02	0.92 (0.56 to 1.50); 0.73
Second follow-up
Cannabis used weekly or more	32/223 (14)	14/184 (8)		56/333 (17)	24/154 (16)		2.03 (0.82 to 5.04); 0.12	1.10 (0.48 to 2.51); 0.83		1.33 (0.47 to 3.83); 0.58		1.74 (0.65 to 4.67); 0.26		2.27 (0.71 to 7.33); 0.16	2.50 (0.86 to 7.28); 0.09
Truancy in previous 3 months	29/110 (26)	8/88 (9)		29/160 (18)	16/78 (21)		3.58 (1.31 to 9.75); 0.01	0.86 (0.29 to 2.54); 0.78		1.90 (0.62 to 5.83); 0.25		1.62 (0.20 to 12.99); 0.64		3.19 (0.82 to 12.50); 0.09	0.90 (0.10 to 7.88); 0.92
Temporary school exclusion in previous 3 months	17/129 (13)	5/95 (5)		25/185 (14)	15/81 (19)		2.73 (0.75 to 9.92); 0.12	0.66 (0.32 to 1.35); 0.25		3.15 (0.46 to 21.49); 0.23		0.63 (0.23 to 1.76); 0.37		4.35 (0.66 to 28.7); 0.12	0.64 (0.19 to 2.18); 0.46
Contact with police	61/225 (27)	38/180 (21)		135/328 (41)	67/152 (44)		1.39 (0.78 to 2.48); 0.26	0.89 (0.50 to 1.58); 0.68		0.98 (0.44 to 2.15); 0.96		0.73 (0.38 to 1.42); 0.35		0.88 (0.41 to 1.91); 0.75	0.83 (0.41 to 1.64); 0.58
Official warnings or convictions	45/226 (20)	23/182 (13)		95/330 (29)	38/153 (25)		1.72 (0.88 to 3.37); 0.11	1.22 (0.59 to 2.53); 0.58		1.65 (0.74 to 3.72); 0.22		1.36 (0.61 to 3.07); 0.45		1.54 (0.69 to 3.45); 0.28	1.72 (0.71 to 4.18); 0.22
Heterosexual sex before age 16	127/220 (58)	60/180 (33)		167/317 (53)	70/146 (48)		2.73 (1.42 to 5.24); <0.01	1.21 (0.55 to 2.65); 0.63		2.53 (1.09 to 5.92); 0.03		1.04 (0.46 to 2.40); 0.91		3.48 (1.49 to 8.12); <0.01	1.39 (0.59 to 3.31); 0.45
Of those with heterosexual experience, ≥2 sexual partners in previous 3 months	25/147 (17)	9/86 (10)		76/196 (39)	33/80 (41)		1.75 (0.66 to 4.69); 0.26	0.90 (0.56 to 1.45); 0.67		0.76 (0.26 to 2.27); 0.62		1.11 (0.51 to 2.40); 0.78		1.33 (0.35 to 5.10); 0.67	1.05 (0.47 to 2.37); 0.90
Often worry in previous few weeks	186/219 (85)	143/176 (81)		201/321 (63)	98/148 (66)		1.30 (0.82 to 2.07); 0.26	0.85 (0.56 to 1.30); 0.46		1.29 (0.79 to 2.11); 0.29		0.91 (0.60 to 1.39); 0.66		1.77 (0.98 to 3.19); 0.06	0.83 (0.53 to 1.29); 0.40

*Adjusted for cluster only.

†Adjusted for baseline variables: housing type, living in non-paid work household, living in lone parent household, ethnicity, dislike of school, truanting, temporary and permanent school exclusions, heterosexual sexual experience, and alcohol use; also adjusted for age, gender, and pupil referral unit attendance at follow-up.

‡Weighted for missing returns at each follow-up and adjusted for above variables.

In terms of social outcomes, at the first follow-up significantly more young people in the YPDP group than in the comparison group reported truanting in the previous six months; this association remained in all analyses. This was largely explained by increased truanting among young women in the YPDP group in the first year cohort of the project. At follow-up 2, the proportion of the YPDP participants who truanted fell and the difference between the intervention and comparison groups was not significant. At the first follow-up, the number reporting temporary exclusion from school in the previous three months was lower than at baseline, with no significant differences between the YPDP and comparison groups. However, among young women, those participating in the YPDP were significantly more likely to report temporary exclusion from school at follow-up 1. We found no differences overall or by sex in temporary exclusions from school or non-participation in education, employment, or training at follow-up 2. We found no significant differences between the YPDP and comparison groups in the number of participants who reported contact with the police at follow-ups 1 and 2 or reported official warnings or convictions at follow-up 2. No significant differences existed in the proportion of YPDP and comparison participants who reported at follow-up 1 that they expected to have a steady job by age 20. Our exploratory subgroup analyses found no differences in any health or social outcomes by programme implementation (data available on request).

Implementation of the programme will be reported in a separate paper, but to provide context for our findings on outcomes we report the key findings here. YPDP projects had initial challenges—for example, in recruiting staff and participants and modifying their practice to embrace the YPDP approach. By the end of the first year, nearly all projects operated a programme that offered the key YPDP components. Projects were able to recruit, retain, and engage many vulnerable young people in relatively intensive provision for a prolonged period (on average 173 hours over 40 weeks), although the average amount of time young people spent on the YPDP was less than targeted. Projects delivered the YPDP in diverse ways. Several projects responded to pressure to meet targets relating to recruitment, attendance, and retention by cooperating with schools to offer education to disaffected students as an alternative to attending some or all of the normal school day both on and off site.

Education about sex and drugs was delivered to different extents and in varying styles across sites. This was generally delivered by youth workers and aimed to enable participants to make informed decisions to delay sex/refuse drugs or reduce associated risks. Sites diverged as to whether they segregated sessions by age or gender. All sites aimed to refer participants to sexual health services when necessary, but according to monitoring data only six did so (although we suspect that not all referrals were recorded). Some but not all projects distributed condoms. Most young people, staff, parents, and other stakeholders rated the YPDP highly. Young people were especially positive about the activities on offer and their relationships with staff. Staff liked working in a more holistic way with young people and thought that through the YPDP they were offering a better service to their participants.

Discussion

Unexpectedly, our analysis suggested that participation in the YPDP was associated with higher rates of some outcomes than occurred at comparison sites. Among young women, YPDP participants more commonly reported teenage pregnancies, early heterosexual sex, and expectation of becoming a teenage parent, as well as temporary exclusion from school and truancy; these associations remained in all models. Our exploratory subgroup analyses found no explanation for these outcomes in terms of the type of agency or the “dose,” contents, or quality of the programme.

Strengths and weaknesses of the study

Our outcome findings might, at least in part, reflect methodological limitations. In practice, young people were sometimes recruited to our comparison group through use of a slightly more general criterion (“vulnerable young people”) than those used in recruitment to the YPDP (see above). However, our comparison group was recruited from sites matched on deprivation and rates of teenage pregnancy and was not systematically at lower risk at baseline. We adjusted for a broad range of pre-hypothesised potential confounders where these significantly differed overall between arms at baseline, checked whether gender specific baseline differences might account for the associations found, and did analyses incorporating propensity scores for data from follow-up 2. We cannot, however, eliminate the possibility of unmeasured/incompletely measured confounding, although this would have to be large to account fully for the unexpected associations found.

Attrition was a major challenge in this study, and our findings might result from participants at higher baseline risk being followed up more completely in the YPDP group than in the comparison group. However, our results suggest that this was true for only some baseline measures; the pattern was reversed for other measures. Weighting for losses to follow-up did not change our findings. However, differential attrition between arms in terms of unmeasured baseline risk factors might explain our findings. Some caution should also be exercised in interpreting our tests of significance, as we did a large number of these. However, our finding of consistent associations regarding sexual health outcomes among young women indicates that random error may not explain these.

Other limitations are likely to have biased the evaluation towards overestimating the benefits of the YPDP. YPDP sites had successfully tendered to participate in the programme, whereas comparison youth work agencies had tendered but not been chosen. Most YPDP agencies were likely to have been selected because they had better capacity, experience, or preparedness, which might have been expected to produce better outcomes regardless of added value from the YPDP. Our outcomes relied on young people’s self reports, so information bias might have led to some overestimation of the benefits of the YPDP. Furthermore, completion of questionnaires was greater among YPDP participants who spent longer on the programme, possibly leading to overestimates of benefits.

Loss to follow-up, as well as the greater than expected intracluster correlation coefficients (for example, 0.12 for heterosexual sex and 0.08 for teenage pregnancy), reduced our power to detect small associations between participation in the YPDP and our key outcomes. Although this cannot explain our unexpected findings of significant associations between participation in the YPDP and, for example, teenage pregnancy among young women, it might mean that some real associations were not detected. This might be the case, for example, with the non-significant associations between participation in the YPDP and weekly cannabis use and official warnings/convictions at follow-up 2. Finally, although our study involved multiple sites and a large sample of young people across England, its results cannot be assumed to be generalisable to other countries.

Meaning of the study

Our finding of no added benefits for YPDP participants over comparison participants for some outcomes, particularly among young men, is in line with findings from some recent studies of youth development programmes.1 5 6 The lack of added benefit may reflect both the high quality of work under way in some comparison sites and the challenge of bringing about detectable effects on health behaviours and outcomes where these are influenced by profound socioeconomic and educational inequalities, peers, and mass media. Some outcomes, such as boys’ temporary exclusions, were less commonly reported at follow-ups than at baseline, perhaps suggesting that young men in both intervention and comparison groups experienced benefits.

However, higher rates of pregnancy, heterosexual sex, truancy, and temporary exclusion from school among young women in the YPDP group do not tally with previous evaluations of youth development. These might be explained by the above methodological limitations or by some YPDP providers experiencing initial disruption as a result of participation in the YPDP, which may have negatively affected initial outcomes, such as young women’s truancy. However, our exploratory analyses found no evidence that outcomes were better in YPDP providers rated as high quality.

The unexpected sexual health outcomes are unlikely to be attributable to the sex education within the programme, because this was a relatively small and variably delivered component and because of the lack of previous evidence for harms arising from sex education.1 Another possibility arises from the potential effects of targeting young people deemed to be at risk. Previous studies suggest that some interventions targeting people at risk can expose participants to the influence of new peers who are more supportive of or more engaged in behaviours associated with risk, thereby spreading risk.10 11 12 13 Whereas the Carrera programme targeted young people in deprived areas, the YPDP aimed to bring together young people deemed to be at risk of teenage pregnancy, substance misuse, and school exclusion. This was not the case in comparison sites, where youth work did not generally target in this way and where pupil referral units targeted only on the basis of young people’s exclusion from school. Our evaluation of the process did not aim to examine the plausibility of this pathway but did find a few cases of participants experiencing bullying during their participation and a few parents who suggested that bringing together badly behaved children might spread misbehaviour.

Additionally, previous studies of interventions targeting vulnerable youths have suggested that adverse outcomes can arise from “labelling.”14 YPDP participants deemed to be “at risk” may have felt labelled as problematic despite the YPDP’s explicit emphasis on young people’s potential. This may have been the case especially where the YPDP was in effect a form of alternative education. Young women in the YPDP group more often reported that they expected to be a teenage parent, possibly reflecting lowered expectations arising from labelling. However, these are speculations and do not explain why effects seemed to differ between young men and women.

We believe that a combination of reasons best explains our findings. Informed by the precautionary principle in public health,15 we recommend that any future implementation of targeted youth development in the United Kingdom should occur only within the context of a randomised trial and with more emphasis on the definition and consistency of the intervention. Although we cannot be certain whether peer group and labelling effects explain our results, we would suggest that any future implementation should ensure that the intervention does not inadvertently bring participants, in particular young women, into contact with more risky peers. This might be achieved by separating provision for young women and men, or for those of different ages, and targeting participants in terms of social disadvantage (as the Carrera programme did) rather than behavioural risk. To minimise labelling, we recommend that any future youth development should be an addition rather than an alternative to school. We also recommend more attention to tackling the wider socioeconomic and educational influences on young people’s health.16

Youth development delivered by the Children’s Aid Society’s Carrera programme to young people in disadvantaged areas of New York city was effective in reducing teenage pregnancies

Subsequent studies of attempted replications elsewhere in the United States did not find such benefits

A youth development programme was delivered to young people in England targeted as being at risk of teenage pregnancy, substance misuse, and school exclusion

More young women participating in this programme reported teenage pregnancies as well as early heterosexual experience and expectation of teenage parenthood than did comparison participants

Methodological limitations may explain at least some of the effects seen, but plausible causes may involve participants encountering more risk oriented peers and feeling labelled as problematic