Effectiveness and cost-effectiveness of a universal parenting skills programme in deprived communities: multicentre randomised controlled trial.

OBJECTIVE: To evaluate the effectiveness and cost utility of a universally provided early years parenting programme.
DESIGN: Multicentre randomised controlled trial with cost-effectiveness analysis.
SETTING: Early years centres in four deprived areas of South Wales. PARTICIPANTS: Families with children aged between 2 and 4 years. 286 families were recruited and randomly allocated to the intervention or waiting list control. INTERVENTION: The Family Links Nurturing Programme (FLNP), a 10-week course with weekly 2 h facilitated group sessions. MAIN OUTCOME MEASURES: Negative and supportive parenting, child and parental well-being and costs assessed before the intervention, following the course (3 months) and at 9 months using standardised measures.
RESULTS: There were no significant differences in primary or secondary outcomes between trial arms at 3 or 9 months. With '+' indicating improvement, difference in change in negative parenting score at 9 months was +0.90 (95%CI -1.90 to 3.69); in supportive parenting, +0.17 (95%CI -0.61 to 0.94); and 12 of the 17 secondary outcomes showed a non-significant positive effect in the FLNP arm. Based on changes in parental well-being (SF-12), the cost per quality-adjusted life year (QALY) gained was estimated to be £34 913 (range 21 485-46 578) over 5 years and £18 954 (range 11 664-25 287) over 10 years. Probability of cost per QALY gained below £30 000 was 47% at 5 years and 57% at 10 years. Attendance was low: 34% of intervention families attended no sessions (n=48); only 47% completed the course (n=68). Also, 19% of control families attended a parenting programme before 9-month follow-up.
CONCLUSIONS: Our trial has not found evidence of clinical or cost utility for the FLNP in a universal setting. However, low levels of exposure and contamination mean that uncertainty remains. TRIAL REGISTRATION: The trial is registered with Current Controlled Trials ISRCTN13919732.

RCT Entities:   Population Interventions Outcomes

Evidence that group parenting programmes can change parenting practices and are cost-effective in treating conduct disorder and child behaviour problems is strong. There is also good evidence that they are effective in preventing behavioural problems in high-risk groups identified by socioeconomic deprivation, ethnic group and experience of life events.

We evaluated the effectiveness and cost-effectiveness of the Family Links Nurturing Programme (FLNP) in a randomised controlled trial (RCT) in four deprived areas of South Wales.

We tested the hypothesis that randomised allocation to the FLNP would be associated with significant advantages over a waiting list control condition, in terms of parenting and child and parental well-being in the short and medium term.

Our trial has not found evidence of clinical or cost utility for the FLNP in a universal setting.

However, low levels of exposure and contamination led to reduced power to detect effects; combined with issues relating to the application of RCTs in this setting means that uncertainty remains.

Although recruitment needed to be extended both geographically and temporally, the full sample size was recruited and follow-up rates were exemplary.

The most important challenges were well-recognised threats to health promotion and disease-prevention trials; those of poor exposure among the intervention group, contamination of the control group. With regard to programme fidelity, programmes run in one of the allocated sites did not meet the standards for running FLNP in all respects.

A further limitation of our study is the lack of information on the effect of the programme on child quality-adjusted life years.

Introduction

Parenting is an important determinant of health and well-being. It plays a role in determining future mental health,1–4 health-related lifestyles (including healthy eating,5 substance misuse,6 teenage pregnancy7), injury rates,8 aspects of physical health,3 9–12 social competence13 14 and educational achievement.15 16

Evidence that group parenting programmes can change parenting practices and is cost-effective in treating conduct disorder and child behaviour problems is strong.17 18There is also good evidence that they are effective in preventing behavioural problems in high-risk groups identified by socioeconomic deprivation, ethnic group and experience of life events.19–22 Although many parenting programmes have been developed and evaluated,23 24 most of this evidence is related to two programmes: the Incredible Years Programme25 26 and Triple P.27 28

Current UK government policies recognise the need for universal parenting support to complement targeted and indicated approaches29–33 and the English Department for Education is currently piloting the offer of free vouchers for parenting classes (the CANParent initiative) to all parents in three areas of the country.34 Such recommendations derive from observations relating to the prevalence of suboptimal parenting,35 the inefficiency of targeting on the basis of identifiable risk factors36 37 and the potential for realising change in high risk as well as whole population groups by reducing the stigma which may be attached to targeted parenting support.36–38 Given the range and prevalence of health and social outcomes on which parent–child relationships have an influence,2 3 5–16 universal approaches are appealing. However, programmes offered universally may suffer from low recruitment and retention rates39 and in order to keep costs low programmes may be of low intensity.34 Universal programmes can also be challenging to evaluate on several counts. Normal populations may show little change on clinically validated outcome measures, control groups may experience contamination40 and, because the effects of parenting on health and social outcomes appear to be life long, cost-effectiveness, essential to inform decision making, may be difficult to establish.

With the exception of the Triple P,27 28 a suite of parenting interventions which can be offered in combination or singly, trials of universal parenting programmes are relatively rare. Two brief group-based parenting programmes have been subject to randomised controlled trials (RCTs); the first a cluster trial in Australia with parents of 8-month-olds41 and the second an individually randomised trial in Germany with preschoolers.42 In these trials, although there has been some indication of changes in parental attitudes, no changes in parenting or child outcomes have been observed. In contrast, trials of Triple P have been successful in showing effects on parenting and child behavioural outcomes including child abuse. Various combinations of universal approaches to Triple P have been subject to large quasi experimental designs at area level43–45 and smaller studies with individual randomised designs.46–48

The Family Links Nurturing Programme49 (FLNP) has much in common with Incredible Years and the group-based component of Triple P and, like these programmes, meets the criteria for effective parenting programmes specified in the recently updated National Institute for Health and Care Excellence Guidelines for the prevention and treatment of conduct disorder.50 Like Incredible Years and Triple P, FLNP not only covers the principles of positive behaviour management but also addresses the emotional underpinning of problem behaviour and the effect on parenting of the parents’ own childhood experiences. It therefore focuses as much on the child's and parent's emotional and social well-being as it does on problem behaviour. The evidence base for FLNP includes qualitative research showing that parents recruited through schools value the programme and perceive it to have an impact on family relationships, children's behaviour and their own mental health51; ‘before and after’ studies in community groups showing impact on self-report measures of relationship quality and well-being52; and routine evaluation by parents attending programmes showing that the great majority value the programme.53

We evaluated the effectiveness and cost utility (as part of a cost consequences analysis) of the programme in an RCT in four deprived areas of South Wales. We tested the hypothesis that participation in the parenting intervention would be associated with significant advantages over a waiting list control condition, in terms of parenting and child and parental well-being in the short and medium term.

Methods

This pragmatic trial was designed in accordance with the consolidated standards of reporting trials guidelines54 and was conducted by a research team independent of the programme, with an independent trial steering committee including chairman, parental representation and members with expertise in statistics; early years’ intervention studies; and child psychiatry. We obtained informed consent from all participants before randomisation and data collection.

Study population

This trial took place between December 2008 and January 2011 at four sites (Cardiff, Newport, Torfaen and Caerphilly) in South Wales. Parents with children aged 2–4 years living in the catchment area of ‘Flying Start’ early years centres who had not previously attended an FLNP were eligible for recruitment.

In total, 1323 families were identified as potentially eligible and approached by Flying Start practitioners who offered a participant information leaflet and put interested families in contact with the research team for more information. Recruitment was in five phases tied to the start time of each course: January, April and September 2009 and January and May 2010. Of families approached, 27% (286) were recruited; two short of the planned sample size of 288. One parent was mistakenly randomised before consent, so the CONSORT flowchart shows a total of 287 families randomised (figure 1). Recruitment rates improved during the trial.55 Retention to follow-up of 89% at 3 months and 84% at 9 months exceeded pre trial estimations.

Figure 1

CONSORT flow chart.

Intervention

FLNP49 is a structured, manualised course comprising of an introductory ‘coffee morning’, followed by 10 weekly 2 h sessions for groups of 6–10 parents which aims to help parents understand and manage feelings and behaviour, improve relationships at home and in school, improve emotional health and well-being and develop the self-confidence and self-esteem which are essential for effective parenting and learning. It offers support to help parents build on pre-existing parenting skills, use positive behaviour management, communication and relationship strategies and look after their own emotional needs, so they can parent more effectively.

The programme is run by a minimum of two facilitators trained over a 4-day programme with a refresher day 1 year later. As part of the contract provider, organisations agree to offer two supervision sessions to facilitators during the course of each programme. The programme provides experiential learning through the use of guided discussion and role play and a copy of the programme book, the ‘Parenting Puzzle’.49 Parents try out new skills at home each week and report back on the achievements they made at the next session. The programme was delivered as per usual practice in these localities with no additional training or supervision. It was delivered in term times and entry to the group was closed after the third session In order to deliver viable groups, parents who were not taking part in the trial attended groups with parents who were. Families randomised to the control arm of the trial were offered usual practice, including advice and other forms of support available in the locality during the trial period. Participants agreed at recruitment not to attend the FLNP until after the 9-month follow-up data collection period was complete if randomised to the control arm.

Outcome measures

All measures were collected during a home visit to the families at baseline (within 2 months of the start of the programme) and at 9 months from baseline (ie, 6 months postcompletion of intervention).55 In addition, self-completion questionnaires covering the parent/self-report outcomes were collected at baseline, 3 months and 9 months.55 Data on the resources associated with the implementation of FLNP were collected from structured interviews with key staff at each of the four study sites, collection of financial information at each site (eg, estimates of room hire and crèche facilities) and discussions with the main trial team.55

Primary outcome measure

The primary outcome was a composite index providing two scales representing negative parenting and supportive parenting measured at baseline and 9 months. These scales were the same as the parenting outcomes previously used in a national evaluation of an early years programme in England (Sure Start)56 except that we substituted an adapted Mothers’ Object Relations Scales (MORS)57 for the Pianta Child Parent Relationship Scale.58 Our measure was thus derived from observations of maternal and child interactions (tone of voice, responsiveness, affection, spontaneous praise and scolding or hitting) made during the researcher visit and scored dichotomously, questions relating to physical punishment and observation of the orderliness of the home all recorded according to the HOME inventory,59 and responses to the MORS adapted by substituting ‘child’ for ‘baby’ collected at the same visit.57 The latter scale produced two factors (warmth and invasiveness) corresponding closely to the conflict and closeness scales on the Pianta scale used in Sure Start. Factor analysis of the HOME observations produced two scales as in Sure Start—responsivity and acceptance. Overall factor analysis showed very similar findings to those derived in Sure Start. Given the much larger sample in the Sure Start study, variables in this study were weighted with Sure Start weightings:—negative parenting−0.06, −0.09, 0.70, 0.60, 0.80 and −0.53, for responsivity, acceptance, harsh discipline and chaos HOME scores and invasiveness and warmth from the MORS, respectively; supportive parenting 0.80, 0.69, 0.20, −0.24, −0.14 and 0.39, for responsivity, acceptance, harsh discipline and chaos HOME scores and invasiveness and warmth from the MORS, respectively. Both negative and supportive parenting scales were approximately normally distributed.

Secondary outcome measures

A range of secondary outcome measures was used to capture changes in health and well-being in parents and children. To measure changes in the child's behaviour, we used the preschool version of a clinically validated measure of childhood emotional and behavioural problems the Parent Account of Child Symptoms (PrePACS).60 Data on positive and negative interactions were gathered using a video of a child's mealtime coded according to the Mellow Parenting Scheme61 and a 5 min speech sample capturing parents’ descriptions of their children and their relationship with each child, coded according to the warmth of their initial statement and the percentage of negative comments about the child.62 Speech samples and videos were coded following the training of researchers by the developer according to the developer's manuals.61 62 Researchers received training in the administration of the PrePACS from the developer.

To measure the child's well-being, we used the PedsQL: parent report,63 a generic health-related quality-of-life measure. Parental well-being was assessed using the Warwick-Edinburgh Mental Well-being Scale (WEMWBS),64 Parenting Stress Index65 and the SF-12,66 from which SF-6D scores were generated to derive utility values for the estimation of the cost utility of FLNP.67 These questionnaires were collected pre intervention, immediately postintervention (3 months), and 9 months (6 months postprogramme).55

Economic evaluation

A cost utility analysis (CUA) was undertaken from the perspective of the UK NHS and Social Services. The cost of implementing FLNP was used in conjunction with differences in quality-adjusted life years (QALYs) generated from the SF-6D utilities derived from the SF-12,66 67 collected from the parents only at baseline and 9 months. The utilities were assumed to remain constant over time, with missing values imputed using predictive mean matching and complete cases only to generate estimates of the cost per QALY gained from FLNP over 5 and 10-year time horizons, based on linear extrapolation of effect over time. A series of sensitivity analyses was undertaken to assess the impact of changes in cost, removing the assumption of benefits remaining constant over time to a scenario that benefits would gradually reduce to zero over time and using imputed values of effect.

Data were collected on costs of implementing the FLNP in the categories of facilitator and administration costs, training, venue hire, crèche provision, course materials and refreshments. The grade and mix of staff used to deliver FLNP programmes varied within and between each study site and so costs were estimated for each site and the mean cost per programme and per family attending derived from these estimates. Implementation costs associated with the programme were estimated by multiplying resource usage by unit costs (where available) sourced from published 2009 PSSRU reference costs for health and social care68 or from local costs provided during interviews.

Programme fidelity

A senior practitioner from Family Links assessed the fidelity of programme delivery using video recordings of three randomly selected sessions for each programme coded against a structured tool which scores sessions from 1 (low fidelity) to 4 (high fidelity) on nine dimensions. One session out of the three selected sessions was watched in its entirety; if this reached a satisfactory standard, the practitioner watched 15–30 min of the other two sessions to ensure consistency. If the first session was not satisfactory, then the other two sessions were assessed in their entirety.

Sample size

We aimed to recruit 144 families into each of the control and intervention groups (288 in total). Assuming a drop-out rate from the trial of approximately 20%; this sample size was sufficient to detect a standardised effect size of 0.4 at 85% power and α of 0.05 in the primary outcome measure if there was no clustering and a standardised effect size of 0.6 allowing for clustering by course with an intraclass correlation coefficient of 0.178 or less; for this sample size calculation, we conservatively imagined clustering within control families as well as intervention families. As the sample size was specified in terms of the number of families rather than the number of children, correlation between siblings in the same family was not considered at the design stage. No interim analyses were planned or conducted.

Randomisation

We stratified randomisation by site and maintained balance between groups within each stratum using a minimisation method using computer-generated random numbers to give an equal chance of allocation to each group when groups within the stratum were balanced and an increased chance (0.667) of allocation to the smaller group when there was an imbalance within the stratum.69 The randomisation service at Warwick Clinical Trials Unit generated the random allocation sequence and informed the FLNP co-ordinators of group allocation following recruitment of a family by the researchers. This ensured the concealment from both families and researchers prior to initial consent being given and from the researchers throughout the trial.55

Blinding

The participants and FLNP coordinators knew the allocation to intervention or waiting list control. The researchers were blind to group allocation, except on rare occasions when families disclosed their assignment status during postcourse interviews. Researchers who did not know the families coded videos and speech samples and all data analyses were carried out blind to group allocation.

Analysis strategy

The primary analysis for the primary as well as secondary outcome measures was carried out by treatment allocated. Positive change scores indicate a benefit to parenting or well-being. A per protocol analysis compared outcomes for families in which parents did not attend an FLNP session to outcomes for families in which parents attended six or more FLNP sessions. For both treatment allocated and per protocol analysis, we report complete cases analysis results for all outcomes. In all analyses, we fitted multilevel models with a random effects term for course and for outcomes corresponding to individual child data and a random effects term for family. For the families in the control, the between-family variance was assumed to be equal to that between families in the intervention group attending different courses; terms in the model were thus included as if each control family formed their own course, so that they contributed to estimating the between course random effect variance. For sensitivity analysis, we performed multiple imputation using predictive mean matching70 and results are reported for the primary outcomes and SF-12. For each outcome, we created 10 imputed datasets using the Hmisc package in the R statistical programme and combined the analysis results from the 10 imputed datasets using Rubin rules.70

Results

The groups had similar baseline characteristics in terms of gender, employment, ethnicity, parents’ education and parenting scores (table 1).

Table 1

Baseline characteristics of families

Characteristic	Control	FLNP
Number of parents	143	143
Number of children	145	150
Negative parenting: mean (SD)	11.07 (10.20)	10.61 (11.36)
Supportive parenting: mean (SD)	26.00 (2.64)	25.88 (2.92)
Parent gender: n (%) of females	139 (99.3)	136 (95.1)
Number of parents with partner: n (%)	49 (34.3)	46 (32.4)
Parent working: n (%) working	112 (78.3)	112 (78.9)
Parent's ethnicity: n (%)
White British	103 (72.0)	100 (69.9)
White other	21 (14.7)	22 (15.4)
Black African	4 (2.8)	0
Black Caribbean	0	1 (0.7)
Asian Indian/Pakistani/Bangladeshi	9 (6.3)	6 (4.2)
Asian Chinese	0	2 (1.4)
Other	6 (4.2)	12 (8.4)
Parent's education: n (%)
NVQ	26 (18.4)	28 (20.1)
GCSE/O levels	46 (32.6)	39 (28.1)
A levels	16 (11.3)	15 (10.8)
Degree	17 (12.1)	15 (10.8)
Higher degree	4 (2.8)	3 (2.2)
None	24 (17.0)	24 (17.3)
Other	8 (5.7)	15 (10.8)

FLNP, Family Links Nurturing Programme.

Uptake of intervention

Forty-eight families (34%) randomly allocated to the intervention (FLNP) group did not attend any sessions, a further 27 (19%) discontinued the programme in the first three sessions. Fifteen control group families (10%) attended an FLNP before 9-month follow-up and a further 13 (9%) control group parents attended other parenting support programmes during the trial.

Main findings: effectiveness

Although both groups improved over time, and the intervention group more so than the control group, the difference in change between groups was not statistically significant at 9 months (change in negative parenting: control group +1.41, 95% CI −0.47 to 3.30; FLNP group +2.31, 95% CI 0.21 to 4.41; difference in change +0.90, 95% CI −1.90 to 3.69; change in supportive parenting: control group +0.38, 95% CI −0.15 to 0.91; FLNP group +0.55, 95% CI −025. to 1.12; difference in change, +0.17, 95% CI−0.61 to 0.94). This pattern of results was repeated in 12 of the 17 secondary outcomes: SF-12; PedsQL; WEMWBS; PrePACS hyperactivity, conduct and internalising (frequency); speech sample warmth of opening statement and video proportion of negative interactions, rate of negative interactions (length), rate of positive interactions (length), rate of negative interactions (involvement) and rate of positive interactions (involvement; table 2). The results for the primary outcome measures and SF-12 by treatment as allocated after multiple imputation were also not statistically significant (table 3). At 3 months, the results were similar with no statistically significant differences in change, although observed outcomes favoured the intervention group for three of the four secondary outcomes (table 4). Per protocol analyses reflected these results at both time points (tables 2 and 4).

Table 2

Outcomes at 9 months: by treatment as allocated and per protocol

	Control			Intervention
Outcomes	n*	Baseline score (SD)	Change at 9 months (SE)	n*	Baseline score (SD)	Change at 9 months (SE)	Effect size (95% CI), p value† analysis by treatment allocated	Effect size (95% CI), p value †‡per protocol analysis
Primary outcomes
Negative parenting	106	11.04 (10.27)	1.41 (0.96)	115	10.72 (11.62)	2.31 (1.02)	0.90 (−1.90 to 3.69), 0.52	1.52 (−1.58 to4.61),0.34
Supportive parenting	106	26.04 (2.75)	0.38 (0.27)	115	25.82 (3.00)	0.55 (0.28)	0.17 (−0.61 to 0.94), 0.67	0.43 (−0.44 to 1.30), 0.33
Secondary outcomes
SF-12	110	0.67 (0.09)	0.01 (0.01)	121	0.68 (0.08)	0.02 (0.07)	0.01 (−0.01 to 0.02), 0.60	0.01 (−0.02 to 0.03), 0.69
Parenting Stress Index	99	123.33 (21.49)	5.88 (1.78)	106	124.13 (22.39)	5.08 (1.72)	−0.80 (−5.74 to 4.14), 0.75	2.88 (−2.78 to 8.54), 0.32
PedQL	112	58.60 (8.90)	−0.20 (0.81)	118	58.51 (7.87)	1.30 (0.80)	1.50 (−0.74 to 3.74), 0.19	1.10 (−1.37 to 3.56), 0.38
WEMWBS score	113	40.61 (9.01)	2.53 (0.81)	121	39.54 (9.71)	3.03 (0.83)	0.49 (−1.82 to 2.80), 0.67	2.02 (−0.56 to 4.59), 0.13
PrePACS
Hscale (weekly)	114	11.70 (6.68)	2.57 (0.66)	129	12.14 (6.84)	3.43 (0.63)	0.86 (−0.95 to 2.66), 0.35	1.61 (−0.46 to 3.67), 0.13
Hscale (typical)	115	13.78 (7.02)	2.62 (0.69)	129	13.41 (7.49)	2.58 (0.82)	−0.03 (−2.18 to 2.12), 0.98	−0.06 (−2.29 to 2.18), 0.96
Cscale (weekly)	116	16.09 (8.38)	1.23 (0.83)	128	16.07 (9.45)	1.27 (1.01)	0.03 (−2.57 to 2.64), 0.98	1.94 (−0.77 to 4.65), 0.16
Cscale (typical)	116	16.58 (8.39)	1.50 (0.79)	127	16.07 (9.94)	1.38 (0.88)	−0.12 (−2.47 to 2.24),0.92	1.29 (−1.27 to 3.84),0.32
Internalising (Freq)	110	5.66 (3.43)	0.26 (0.33)	122	5.44 (3.19)	0.40 (0.34)	0.13 (−0.82 to 1.09), 0.78	−0.08 (−1.16 to 1.00), 0.89
Internalising (reassure)	110	5.46 (3.73)	0.39 (0.42)	126	5.18 (3.68)	0.10 (0.40)	−0.29 (−1.43 to 0.85), 0.62	−1.02 (−2.34 to 0.30), 0.13
5 min speech sample (warmth of opening statements)	107	6.30 (1.38)	0.29 (0.14)	124	6.48 (1.30)	0.30 (0.14)	0.01 (−0.39 to 0.40), 0.97	0.14 (−0.30 to 0.58),0.54
5 min speech sample (% negative comments)	105	0.43 (0.26)	0.07 (0.03)	117	0.41 (0.27)	0.05 (0.03)	−0.01 (−0.10 to 0.07), 0.75	0.06 (−0.03 to 0.15), 0.21
Meal-time video
Proportion of negative interactions	84	0.12 (0.16)	−0.02 (0.02)	87	0.14 (0.15)	0.03 (0.02)	0.05 (−0.01to 0.11), 0.13	0.03 (−0.04 to 0.10), 0.41
Rate of negative interactions (length)	84	0.49 (0.73)	−0.02 (0.10)	87	0.52 (0.72)	0.09 (0.10)	0.11 (−0.19 to 0.40), 0.46	0.09 (−0.23 to 0.41), 0.59
Rate of positive interactions (length)	84	3.65 (2.57)	−0.14 (0.46)	87	3.13 (2.24)	0.62 (0.46)	0.75 (−0.53 to 2.04), 0.25	0.56 (−0.87 to 1.99), 0.44
Rate of negative interactions (involvement)	72	0.68 (0.96)	−0.19 (0.19)	73	0.72 (0.86)	−0.16 (0.19)	0.03 (−0.50 to 0.57), 0.91	0.07 (−0.52 to 0.66), 0.82
Rate of positive interactions (involvement)	72	4.99 (3.19)	0.93 (1.01)	73	4.66 (2.87)	2.55 (1.01)	1.62 (−1.21 to 4.45), 0.26	1.16 (−1.98 to 4.29), 0.47

*Corresponds to number of children included in the analysis.

†The effect size is the difference between difference in control and intervention groups with a positive value indicating that the intervention is superior to control and vice versa.

‡The per protocol analysis gives the results comparing outcomes for families that did not attend any FLNP session to outcomes for families that attended six or more sessions. Group numbers varied for measures: negative parenting, control group, n=128, intervention group n=62; supportive parenting, control group n=128, intervention group n=62; SF-12, control group n=134, intervention group n=62; Parenting Stress Index, control group n=120, intervention group n=56; PedQL, control group n=133, intervention group n=68; WEMWBS score, control group n=136, intervention group n=64; Hscale (weekly), control group, n=140, intervention group n=68; Hscale (typical), control group, n=141, intervention group n=68; Cscale (weekly), control group n=142, intervention group n=68; Cscale (typical), control group n=142, intervention group n=67; internalising (Freq), control group n=133, intervention group n=65; internalising (reassure), control group n=138, intervention group n=66; 5-min speech sample (warmth of opening statements), control group n=129, intervention group n=69; 5-min speech sample (% negative comments), control group n=124, intervention group n=68; proportion of negative interactions, control group n=94, intervention group n=55; rate of negative interactions (length), control group n=94, intervention group n=55; rate of positive interactions (length), control group n=94, intervention group n=55; rate of negative interactions (involvement), control group n=78, intervention group n=48; rate of positive interactions (involvement), control group n=78, intervention group n=48.

FLNP, Family Links Nurturing Programme; PrePACS, preschool Parent Account of Child Symptoms; WEMWBS, Warwick-Edinburgh Mental Well-being Scale.

Table 3

Outcomes at 9 months: by treatment as allocated after multiple imputation

Outcomes	n (control)	n (intervention)	Effect size (95% CI), p Value
Negative parenting	145	150	1.10 (−0.66 to 2.86), 0.22
Supportive parenting	145	150	0.35 (−0.14 to 0.83), 0.16
SF-12	143	143	<0.01 (−0.02 to 0.02), 0.81

Table 4

Outcomes at 3 months: by treatment as allocated and per protocol

	Control			Intervention
Outcomes	n*	Baseline score (SD)	Change at 9 months (SE)	n*	Baseline score (SD)	Change at 9 months (SE)	Effect size (95% CI), p value† analysis by treatment allocated	Effect size (95% CI), p value †‡ per protocol analysis
SF-12	115	0.66 (0.09)	0.01 (0.01)	125	0.68 (0.08)	0.01 (0.01)	−0.01 (−0.02 to 0.02), 0.88	−0.01 (−0.03 to 0.01), 0.38
Parenting Stress Index	108	123.12 (22.22)	0.99 (1.65)	103	124.55 (23.37)	3.99 (1.69)	3.00 (−1.66 to 7.66), 0.21	−1.27 (−6.69 to 4.14), 0.64
PedQL	128	58.44 (9.06)	−0.48 (0.77)	116	59.15 (7.13)	0.21 (0.83)	0.69 (−1.55 to 2.92), 0.55	1.31 (−1.28 to 3.89), 0.32
WEMWBS score	128	40.70 (9.45)	0.85 (0.78)	117	39.85 (9.88)	2.26 (0.99)	1.41 (−1.12 to 3.93), 0.27	1.11 (−1.60 to 3.82), 0.42

*Corresponds to number of children included in the analysis.

†The effect size is the difference between difference in control and intervention groups with a positive value indicating the intervention is superior to control and vice versa.

‡The per protocol analysis gives the results comparing outcomes for families that did not attend any FLNP session to outcomes for families that attended six or more sessions. Group numbers varied for measures: SF-12, control group n=145, intervention group n=63; Parenting Stress Index, control group n=125, intervention group n=57; PedQL, control group n=146, intervention group n=68; WEMWBS score, control group n=149, intervention group n=62.

FLNP, Family Links Nurturing Programme; WEMWBS, Warwick-Edinburgh Mental Well-being Scale.

Overall fidelity score for courses in site A was 87%, site B 82%, site C 72% and site D 60% (table 5). Fidelity in phase 1 courses was 85%, phase 2 83%, phase 3 75%, phase 4 77% and phase 5 47%. Fidelity was much lower in phase 5 due in part to the higher proportion of families from site D in this phase of the trial. The per protocol analysis was repeated comparing the primary outcome measures for control families with families attending six or more FLNP sessions on a course with fidelity of 80% or more. The estimated differences and between those who attended at least six FLNP sessions with high fidelity (n=41) and control groups (n=128) on supportive and negative parenting and 95% CI were 0.45 (−0.50 to 1.41) and 0.00 (−3.45 to 3.45), respectively. Neither difference was statistically significant.

Table 5

Overall fidelity score distributions

	Percentage of programme scoring at each fidelity rank
Fidelity ranking 1 low, 4 high)	1	2	3	4
Welcome	2	14	38	46
Family feedback	4	17	44	35
Topics before break	6	21	40	33
Topics after break	5	26	43	26
Time to have a go	0	22	22	56
Modelling group rules	4	25	38	33
Active listening	0	40	43	17
Empathy	4	25	33	38
Nurturing environment	4	18	31	47

Economic evaluation results

Costs of the delivery of FLNP varied between sites (table 6) due to difference in staff delivering the programme—the implications of which are explored in the sensitivity analysis. The cost per family per programme (£648) was used as the cost differential between FLNP and control group for the baseline findings. Cost per QALY for FLNP, based on the complete case analysis, was estimated at £34 913 over 5 years and £18 954 over 10 years duration, with the probability that FLNP can be regarded as cost-effective as 36% against the £20 000 threshold over 5 years, and 51% over 10; and 47% against the £30 000 threshold over 5 years, and 57% over 10 (table 7).

Table 6

Summary of costs associated with implementation

Total costs	Site A	Site B	Site C	Site D	Total
Salaries*	£63 002	£33 341.4	£52 772	£44 423	£193 538
Training†	£5801.90	£4733.90	£5081.90	£5144	£20 761.70
Room hire‡	£3883.89	£3003	£3000	£3000	£12 886.89
Course materials‡	£3750	£1625	£2250	£2250	£9875
Crèche‡	£17 862.66	£17 097.96	£15 000	£15 000	£64 960.62
Total cost	£94 300.45	£59 801.26	£78 103.90	£69 827	£302 022.21
No of courses	15	13	15	15	58
Cost per 11-week programme	£6286.70	£4600	£5206.93	£4654.47	£5207.28
Mean cost per 11-week programme					£5187.03 (SD=782.71)
Mean cost per family per programme (based on 8 parents attending)					£648.38 (SD=97.84)
Cost per family per programme (based on 13 parents invited to attend)					399
Cost per family per programme (based on 6 parents completing the programme)					865

*We applied hourly rates based on grade of staff (including on-costs) from PSSRU52; 34/h for health visitor; 23/h for parenting coordinator and 18/h for community support worker. Administrative overheads were estimated at 10% direct salary costs in line with recommendations from the local finance teams.

†Training was based on cost of facilitator (34/h for health visitor) and hourly rate for staff to attend initial training from PSSRU68 and top up sessions including travel costs based on HM revenue and customs approved mileage rates.73

‡Costs were derived directly from financial records and interviews with staff at local sites.

Table 7

Cost utility analysis

	FLNP	Control	Difference
Cost per programme (£) 8 participants	648	0	648
Cost per programme (£) 6 participants	895	0	895
Cost per programme (£) 13 participants	399	0	399
Complete cases
QALY gain (5 years)	0.053	0.035	0.019
QALY gain (10 years)	0.098	0.064	0.034
Cost per QALY gained (5 years) (£)	34 913		Probability ≤£20 000=36%
			Probability ≤£30 000=47%
Cost per QALY gained (10 years) (£)	18 954		Probability ≤£20 000=51%
			Probability ≤£30 000=57%

FLNP, Family Links Nurturing Programme; QALY, quality-adjusted life year.

Sensitivity analyses show that the impact of cost is highly relevant in determining the likelihood that FLNP can be viewed as representing value for money. For example, the distribution of implementation costs across a larger number of attendees (cost per family of £399 when 13 parents attending) increases the likelihood that the programme would be cost-effective given a £20 000 threshold from 36% to 49%—using a 5-year time horizon; whereas a smaller number of attendees (cost per parent of £865 when six parents attending) reduce the likelihood to 26% with a 5-year time horizon.

Relaxing the assumption that benefits remain constant over time and assuming that the effect diminishes to zero by the end of the time period considered results in an estimated cost per QALY of £56 885 for the 5-year duration (probability cost-effective at £20 000 = 30%) and £29 664 for the 10-year time horizon (probability at the £20 000 threshold=44%; table 8).

Table 8

Sensitivity analysis

	Cost per QALY (5 years (£)	Probability cost effective	Cost per QALY (10 years (£)	Probability cost effective
Baseline	36 000	35% @ £20 000	19 115	51% @ £20 000
		46% @ £30 000		58% @ £30 000
Lowest cost	21 485	49% @ £20 000	11 664	58% @ £20 000
		55% @ £30 000		62% @ £30 000
Highest cost	46 578	26% @ £20 000	25 287	45% @ £20 000
		40% @ £30 000		53% @ £30 000
Benefits diminish to zero over time	56 885	30% @ £20 000	29 664	44% @ £20 000
		40% @ £30 000		50% @ £30 000
Imputed values	34 122	37% @ £20 000	18 525	51% @ £20 000
		47% @ £30 000		58% @ £30 000

QALY, quality-adjusted life year.

Discussion

This trial investigated the effectiveness of an existing parenting programme offered on a universal basis in deprived areas in the context of an open access early years programme offering a wide range of support for parenting. Although recruitment needed to be extended both geographically and temporally,55 the full sample size was recruited and follow-up rates were exemplary.

The results of the treatment allocated analyses showed no statistically significant difference between intervention and control groups on any of the outcome measures. For most outcomes, improvement was observed in both groups with families in the FLNP arm of the trial improving more than those in the control arm by a small and not statistically significant amount, the differences observed being very much smaller than those which the trial was powered to detect.

The 3-month results suggest that differences in parental well-being were greatest at the end of the programme and declined over the next 6 months. In contrast, the observed differences in child well-being as measured by the PedsQL increased over time.

Our economic analyses do not provide evidence that FLNP represents value for money, but are based on an estimate of parental utility alone. A recent study of the cost-effectiveness of the Incredible Years Programme in a targeted population on key outcomes in later life has demonstrated favourable long-run economic returns.71

Limitations of the study

The most important challenges were well-recognised threats to health promotion and disease-prevention trials; those of poor exposure among the intervention group, contamination of the control group and varying programme fidelity. Our predicted programme non-attendance and dropout rate in the first three sessions, based on the provision of this programme in other settings,52 52 was 16%. In the event, the rate was 53%. It is possible that the trial setting may have influenced attendance rates by preferentially recruiting families who were not ready to change40 as such families were more likely to be willing to be randomised. Testing the effectiveness of an existing programme in a pragmatic trial makes it challenging to restrict access to the programmes in control group families and in this trial 10% of the control group violated the rules of the trial and attended an FLNP course before the 9-month follow-up. A further 9% of control parents accessed different parenting support courses available in the locality, but only one of the intervention group families accessed any other type of parenting support. The main trial results therefore compare the changes in parenting and family well-being among a group in which just under one in two parents were exposed to the FLNP with changes among a control group in which roughly one in five parents attended the FLNP or other parenting support programme. Had these factors been entirely responsible for the trial findings, however, we would have expected to see significant results in the per protocol analysis. Changes in the latter analysis were larger than those in the as-allocated analysis, but still did not reach statistical significance.

Our assessment of programme fidelity was constrained by costs. We showed that programmes run in one of the allocated sites did not meet the standards for running FLNP in all respects. It is important to note that the sites running programmes with the highest level of fidelity were those with the highest costs and those with the lowest fidelity, the cheapest to run. We attempted an analysis of results by programme fidelity and also by area, and no statistically significant differences between the groups were found. It may be that our estimates of fidelity were inadequate being based on a random sampling of only one session in most of the programmes.

A further limitation of our study is the lack of information on the effect of the programme on child QALYs. FLNP aims, by improving parenting, to improve children's well-being as well as parent's well-being. In our CUA, we were able only to include change in parental QALYs and the measure used (SF-6D) did not prove sensitive to variation in parental well-being as measured by other instruments. Given that non-significant positive changes in PedsQL scores were considerably greater (>1 SE) than the non-significant changes in parental SF-6D scores, including child QALYS could greatly increase cost utility estimates. If it becomes possible in future to estimate QALYs from PedsQL and adopt sensitive measures of parental well-being, it will be possible to revisit these estimates. If, as has been suggested, parenting has a life-course impact on health and social outcomes the relatively short-term outcomes possible in RCTs are inadequate to assess the full benefits and cost savings.

Despite inclusion in data collection, little self-reported information was provided by parents (and verified by contact with the research team) at follow-up on the use of health, social service, educational psychology or criminal justice service contacts during the course of the trial, attributable to these events or to involvement in the FLNP, with insufficient data to undertake any meaningful analysis. Several of these services are, however, unlikely to be relevant to children of 2–4 years. This has limited the ability of the study to consider the full range of perspectives in the analysis.

Meanings and implications of the study

Although outcome measures improved more in the FLNP group than in the control group, the results of this trial fail to show that FLNP improved parenting or child or parental well-being more than could be expected by chance and do not provide evidence that FLNP represents value for money. Three main reasons for these negative results warrant consideration. The first is that the FLNP may not be effective in changing parenting in a universal setting. It would appear that programmes like Triple P which focus primarily on behaviour management are effective in a universal setting and this may simply be a better approach. Triple P is a more costly programme than FLNP. It can be offered more intensively with five different levels of intervention for different levels of need, including universal media coverage, and well-resourced trials have been able to offer the programme in this way. FLNP is a stand-alone group-based programme which may work synergistically with other parenting interventions including media coverage. In this trial, both intervention and control group parents were exposed to general parenting interventions available in the Flying Start setting.

FLNP aims to address the promotion of mental well-being in parents and children as well as behaviour management, thus potentially influencing resilience to a variety of mental health problems including anxiety and depression throughout the life course. In this way, it does more to address the aims of the current government strategy No Health without Mental Health32 than the better researched behaviour management programmes. However, as we failed to identify any change in parenting or family well-being as a result of the trial, it may be that the contents and approach of the programme need revisiting.

The second reason why the trial may have produced negative results is that the programme was implemented poorly in this setting. Even Triple P has shown negative results in independent pragmatic trials72 due in all likelihood to inadequate implementation. This trial measured changes due to the FLNP in an established setting where no attempt was made to improve implementation for the trial. It may be that the training and supervision of these facilitators were not sufficient to maintain high-quality provision. Our assessment of fidelity was constrained by resources. Although it was judged as adequate by the provider in three of the four settings, we did not examine every session and would not have known if problems had arisen in other sessions. We were also unable to assess the quality of supervision offered to facilitators by the provider organisations.

Finally, the trial may have produced negative results because of the research design.

It is possible that the outcome measures we selected were not sensitive to changes which did occur. The observations in the HOME inventory, the 5 min speech sample and the meal-time video were made over a relatively short visit and some parents refused the video.55 Although objective measures are highly valued, the parenting recorded is that of parents on their best behaviour. Against this, we did not identify changes in parent or child well-being either.

The trial was powered to capture effect sizes reported for parenting trials in general most of which are in clinical or targeted settings where opportunities for improvement are greater than in universal populations. It may be that the trial was underpowered to pick up the level of changes that could represent a valuable change at the population level where smaller differences have great impact because of the numbers involved.

The open access early years centre setting certainly presented challenges including the contamination of the control group due to the ready availability of parenting support, and unusually low levels of attendance at the programme. We have also identified in a separate study40 the possibility of a threat to the external validity of the trial. Parents who consented to randomisation may have been different from those who would normally sign up for the FLNP in that some parents did not recognise a need for the programme for themselves and their families. They may therefore have been less ready to change and indeed had less need to change than those who attend the FLNP in the absence of a trial. Together, these factors undoubtedly reduced the study's power to detect any impact of the FLNP.

Future research

Overall, we have concluded that uncertainty remains about the effectiveness and cost utility of the FLNP in universal settings and that further investigation is required. It may be that, given the multiple challenges faced by trials of this nature, studies need to have more statistical power, be based in settings where contamination can be controlled, employ outcomes that are known to change during the course of the programme and where very high-quality programme implementation can be ensured. In the meanwhile, the results of this trial provide estimates to inform the design of a future trial.