Enhancing the mental health promotion component of a health and personal development programme in Irish schools.

This study set out to examine the impact of a health and personal development programme (the Social, Personal and Health Education Programme) which had been 'enhanced' by the addition of a mental health promotion component. Students aged 12-16 years attending 17 secondary schools were randomly allocated as clusters to participate in either the standard programme (SP) or the enhanced programme (EP). Both programmes were delivered over an eight-month period. Questionnaires assessing help-seeking, emotional and behavioural difficulties and students' views of school social environment were completed by students in both groups at pre-intervention, post-intervention and at six-month follow-up. One thousand and seventy-two students participated. Analyses were carried out on the 782 students (72.9%) for whom data were available at all three time points. Both groups showed improvement over time in terms of emotional and behavioural difficulties, with male students identified as 'at risk' in the EP showing significantly greater improvements than males 'at risk' in the SP. Although there were few differences in outcomes between the groups, males identified as 'at risk' appeared to benefit selectively from the EP.

RCT Entities:   Population Interventions Outcomes

Introduction

Although there is general agreement that schools can play an important role in mental health promotion (American School Health Association, 2008; Kraag, Zeegers, Hosman, & Abusaad, 2006; Sheehan et al., 2002; World Health Association, 1997), the literature on the effectiveness of various interventions to achieve this shows inconsistent results (Jané-Llopis & Barry, 2005; Stewart-Brown, 2006; Weare & Markham, 2005). This is not surprising, given that a diverse range of approaches have been subsumed under the mantle of ‘mental health promotion’. For example, social and personal development programmes may not be considered by some to be effective forms of mental health promotion, while others may consider them to be fundamental in mental health promotion. Also, the boundary between ‘prevention’ programmes and mental health ‘promotion’ programmes is often unclear.

Educators are increasingly exhorted to attend to social and emotional learning, which can lead to schools being targeted with multiple programmes which are often short term and may compete with each other (Payton et al., 2000). This can lead teachers to become disenchanted with the whole concept of mental health promotion. There is now good evidence from a number of systematic reviews of universal approaches to mental health promotion in schools that approaches which fulfil certain criteria show positive evidence of effectiveness and are associated with beneficial effects on a range of mental health, social, emotional and educational outcomes (Durlak, Weissberg, Dymnicki, Taylor, & Schellinger, 2011; Weare & Nind, 2011; Wells, Barlow, & Stewart-Brown, 2003). These criteria include a whole school approach, continuous implementation for more than one year and having as their main focus positive mental health promotion rather than prevention of mental illness. Interestingly, while the systematic review carried out by Wells et al. (2003) identified 425 published studies, only 17 fulfilled their inclusion criteria for further study, reflecting the huge variation in the type and quality of published work on the subject. A review article by Weare and Murray (2004) further emphasized the importance of the multi-dimensional, integrated, whole school approach necessary for effective mental health promotion. Durlak et al. (2011) showed that programmes which fulfilled four criteria were most likely to be associated with positive outcomes. These criteria are captured by the acronym SAFE: the programme must be Sequenced, which means it uses a connected and coordinated set of activities to achieve its objectives relative to skill development; it must be Active, which involves using active forms of learning to help youth learn new skills; it must be Focused, meaning it must have at least one component devoted to developing personal or social skills and it must be explicit, meaning it must target specific social and emotional learning skills rather than targeting skills or positive development in general terms.

Even if these principles are rigorously applied, there is no guarantee of success, as shown by Sawyer et al.'s (2010) evaluation of a school-based programme for the prevention of depression. This project was carried out for three years, with the programme itself taking a whole school approach and with what most people would consider to be a very active mental health promotion programme at its core. However, this study failed to show any difference in outcomes between participants and controls, nor even any difference in school climate.

Social personal and health education in Ireland

Rates of mental health disorders in young people in Ireland are similar to those in other westernized cultures (Lynch, Mills, Daly, & Fitzpatrick, 2006; Martin & Carr, 2006). Suicide rates in young men are of particular concern, being fifth highest in Europe, and the leading cause of death in young men in the age range of 15–24 years (Central Statistics Office, 2009). Child and adolescent mental health services in Ireland are reported to be seriously overstretched, with significant gaps in service provision (Expert Group on Mental Health Policy, 2006). This has strengthened the argument for mental health promotion programmes in schools, and influenced the decision by the Health Research Board, Ireland's national health research funding agency, to fund a study to develop and evaluate a mental health promotion programme for use in Irish secondary schools. The study was carried out in collaboration with the Social, Personal and Health Education (SPHE) Support Service of the Department of Education and Science (DES). This body provides training and support to schools in the provision of the SPHE programme, a health and personal development programme which is an obligatory part of the curriculum in Irish schools, at both primary and secondary level.

The DES lists the following as the aims of the SPHE programme to enable students to develop personal and social skills; to promote self-esteem and self-confidence; to enable students to develop a framework for responsible decision-making; to provide opportunities for reflection and discussion and to promote physical, mental and emotional health and well-being (Department of Education & Science/National Council for Curriculum & Assessment, 2001). Although not officially a ‘mental health promotion programme’, most people would consider that these aims are fundamental to mental health promotion. Schools are encouraged to have a critical mass of teachers trained in the ethos and delivery of the SPHE programme, which is delivered in one class period each week throughout the students’ time in primary school and throughout the first three years in secondary school. Teachers are also encouraged to utilize SPHE ethos and principles in their delivery of other subjects. The SPHE programme contains several models that are relevant to mental health promotion, including ‘communication’, ‘belonging and integrating’, relationships’ and ‘emotional health’. Though implementation and delivery of the SPHE programme do not always live up to the high ideals recommended by the DES, it is incorporated into the curriculum in most schools in Ireland (Geary & Mannix McNamara, 2005).

The present study

In 2002, a group of young people attending a child and adolescent mental health service in Dublin developed a DVD called ‘Working Things Out’, with support from mental health professionals. In it, the young people told their stories with animated figures but with their own voices of how they got through difficult times in their lives when they had to deal with issues such as bullying, depression, self-harm, suicidal behaviour, family breakdown and so on. (Fitzpatrick, Brosnan, & Sharry, 2009). The stories on this DVD have been used to develop a programme to enhance the delivery of the mental health promotion aspects of the SPHE programme. Each story forms the basis of a lesson plan, which has particular learning objectives. These lesson plans have been manualized, and form the basis of the ‘Working Things Out through SPHE’ Teachers’ Resource Book (Power & Cleary, 2007). The stories form the basis for discussion, exercises and role-play. The ‘Working Things Out through SPHE’ programme has been described in more detail in a previous publication (Fitzpatrick et al., 2009).

This study set out to determine whether students participating in SPHE that had been ‘enhanced’ by the Working Things Out programme had better outcomes in terms of help-seeking, emotional and behavioural difficulties and their view of school social environment, than those who participated in ‘standard’ SPHE.

Method

Research design

An initial pilot study of the Working Things Out through SPHE programme showed favourable results, but identified a number of methodological problems (Fitzpatrick et al., 2009). In response to the pilot study, this study utilized a cluster sample-based randomized trial, comparing the enhanced SPHE programme with the standard SPHE programme. Quantitative measures were completed by students before they started the programmes (Time 1), after eight months of programme delivery (Time 2) and six months after that (Time 3).

The standard programme condition

Students in the standard programme (SP) condition participated in the standard delivery of SPHE over an eight-month period coinciding with the school year. This involved trained SPHE teachers delivering ideally one class period per week, following the Guidelines for Teachers published by the DES (Department of Education & Science/National Council for Curriculum & Assessment, 2001). These advise coverage of the 10 modules: belonging and integrating; self-management: a sense of purpose; communication skills; physical health; friendship; relationship and sexuality; emotional health; influences and decisions; substance use and personal safety. The order in which the modules are addressed is flexible, with teachers being encouraged to fit modules to the context of what is happening in the class and school at any particular time. A variety of teaching approaches, focusing on active participation, reflection, discussion and role-play, are used, and learning outcomes are defined for each module. SPHE is a holistic curriculum that aims to provide a vehicle to encourage dialogue and discussion among students and between students and teachers and to foster the students’ ability to think and make judgements.

The enhanced programme condition

Although all teachers were trained in an SPHE delivery, those delivering the enhanced programme (EP) had an additional one-day training course in the use of Working Things Out through SPHE, and access as required to the Project Leader (author MP) throughout the study. Students in the EP condition participated in the delivery of SPHE over an eight-month period as described above, but in addition teachers used nine ‘Working Things Out’ stories to bring the mental health promotion aspects of the standard modules to life. Each story had a structured lesson plan built around it, and these were manualized (Power & Cleary, 2007). For example, the module ‘belonging and integrating’ lends itself to the use of a Working Things Out (WTO) story by a young boy who found the transition from primary to secondary school difficult, having been previously bullied, and how he managed those difficulties and the relief he experienced when he was able to talk to someone about how he felt. The lesson plan around that story started with a group exercise called ‘In Groups and Out Groups’, the teacher then showed the story, following which he/she asked questions to stimulate discussion. Students and the teacher then summarized what was covered, and students completed a ‘Director's Chair’ handout which asked simple questions about the story and what the student took from it. This approach exposed the students to a peer dealing with a common problem, the use of words to express feelings, problem-solving and the concept of telling someone how you are feeling if the going gets too tough. Each of the nine stories was covered in 12 forty-minute class periods over the eight months of the study. Some stories were covered more than once. Teachers were asked to complete a fidelity check after each WTO story class.

Assignment and participants

Sampling began by contacting 44 post-primary schools where there were teachers who had been trained in an SPHE delivery. These schools were located in a specific geographical area which included inner-city pockets of chronic deprivation with high rates of drug abuse and criminality, large local authority housing estates and some more affluent suburbs. Thirty-one schools expressed interest in participation in the study. However, for a variety of reasons (e.g. unavailability of teachers for training and difficulty with timetabling) the final sample comprised 17 schools (39% of possible participating schools).

Having stratified for school type (all male, all female and co-educational), second-year classes (students in the age range of 12–16 years) in these 17 schools were randomly allocated to either SP or EP condition. Students in the SP schools participated in the standard SPHE programme, whereas those in the EP schools participated in the standard SPHE programme which had been enhanced with the ‘Working Things Out through SPHE’ lessons. The final sample comprised two all male schools, three all female schools and three co-educational schools in each group, while the 8 second-year classes in one very large co-educational school were randomly allocated to either the SP or the EP condition.

The parents of all students eligible to take part in the project received a letter from the School Principal informing them of the project and inviting them to an Information Evening. They were asked to discuss the project with their son/daughter and to return an ‘opt out’ form if they did not wish their son/daughter to participate. ‘Opt out’ forms were received for only four students. One thousand and seventy-two students completed data collection at Time 1. Of this sample, 53% (n = 568) were male and 47% were female (n = 504). The group ranged in age from 12 to 16 years (M = 13.6, SD = 0.544). There were 545 (50.8%) participants in the SP condition and 527 (49.2%) in the EP condition.

Measures

A questionnaire pack containing a number of standardized and non-standardized measures was compiled. A cover sheet on the questionnaire packs requested that the students tick their gender and supply their date of birth.

The strengths and difficulties questionnaire

The strengths and difficulties questionnaire (SDQ) is a brief behavioural screening questionnaire that provides balanced coverage of children's and young people's behaviours, emotions and relationships (Goodman, 1997; Goodman, Ford, Simmons, Gatward, & Meltzer, 2000). It asks about 25 positive and negative personal attributes. The 25 items are divided into five scales: conduct problems, hyperactivity, emotional difficulties, peer problems and prosocial behaviour, with five items in each scale. The scores from these scales, excluding the prosocial scale, are summed to generate a total difficulties score. A score 16 or over on the total difficulties scale indicates a young person ‘at risk’ of a mental health disorder. There are two versions of the SDQ – an informant-rated (parent or teacher) and a self-rated version completed by the young person. The self-rated version was used in this study. Goodman (2001) reported the internal reliability of the scale and subscales to be acceptable, with an average Cronbach's alpha of 0.73 across the subscales. The validity of the scale and subscales is discussed by Van Roy, Veenstra, and Clench-Aas (2008).

The children's coping strategies checklist

The children's coping strategies checklist (CCSC) is a 54-item self-rated questionnaire, which provides a measure of children's coping strategies in four domains: support seeking; avoidance; active coping and distraction (Ayers & Sandler, 1996). Active coping consists of using cognitive decision-making, direct problem-solving, seeking understanding and use of positive cognitive restructuring. Distraction coping consists of physical release of emotions and use of distracting actions. Avoidance coping consists of avoidant actions and cognitive avoidance. Support seeking coping includes problem-focused support and emotion-focused support. Children are asked to describe how often they usually use each behaviour when they have a problem (never, sometimes, often and most of the time). Some examples of items from the checklist ask the student to rate on a four-point scale (1 = never, 2 = sometimes, 3 = often and 4 = most of the time) how they coped with problems in the past month, giving them options such as ‘you listened to music’; ‘you tried to ignore it’; ‘you let other people know how you felt’ and ‘you went for a ride on your bike’. Adequate reliability and validity have been documented (Ayers, Sandler, West, & Roosa, 1996; Sandler, Tein, & West, 1994). A study by Chen, Weiss, Heyman, Cooper, and Lustig (2010) using this measure reported the Cronbach's alpha to range from 0.71 to 0.86.

Non-standardized measures

Two additional questionnaires were developed for the study. The first was a help-seeking questionnaire, which asked students from whom they would seek help (friend, parent, teacher, mental health professional, telephone helpline, etc.) when dealing with a personal problem (Conlon, Power, Cleary, & Guerin, 2010). Participants were asked to indicate a response on a Likert scale ranging from 1 (Very Unlikely) to 7 (Very Likely). Participants also had an opportunity to indicate that they would not look for help. A total help-seeking score was generated by recoding the ‘No Help’ response and then by summing all scores. A higher score indicated a greater likelihood to seek help from someone and Cronbach's alpha suggested an acceptable internal reliability (0.686). The second was the ‘What is your school like?’ questionnaire which provided a measure of school social environment or ethos. This questionnaire sought information on students’ perception of the school's social and physical environment, their involvement in the school, their engagement and teacher–student relations, factors which have been shown to contribute to school ethos (Bonell, Fletcher, & McCambridge, 2007; Lee, Chen, & Smerdon, 1996; Markham, Young, Sweeting, West, & Aveyard, 2012; Rutter, Maughan, Mortimore, & Onslow, 1979). It included 10 statements which were selected on the basis of reflecting a supportive school environment (e.g. Students are encouraged to achieve and learn in this school and Students at this school feel a sense of belonging and worth) and participants reported if these were evident in the school in their opinion. Each item was scored 1 ‘No’ or 2 ‘Yes’ and items were summed to give a total score, with higher scores indicating a more supportive environment. Cronbach's alpha suggested very good internal reliability (0.859).

Study hypothesis

The EP had a greater explicit focus on the mental health promotion aspects of SPHE, and students who participated in it had these brought to life by seeing and hearing a similar age young person describing how they coped with a common emotional problem and by the discussion and skills practice elements of the lesson plans delivered around the Working Things Out stories. The study therefore hypothesized that students who participated in the EP would show greater improvement than those in the SP in their prosocial behaviour (SDQ), greater reduction in their total difficulties (SDQ), greater improvement in their use of active coping and support seeking and less use of avoidance (CCSC), and greater improvement in their help-seeking and in their view of school ethos.

Procedures

The study was approved by the Research Ethics Committee of the Mater Misericordiae University Hospital. An ID number was allocated to each student, which enabled tracking of their responses throughout the course of the study, without requiring them to give their names on the questionnaires. The questionnaires were read out to the class group by one of the researchers (AC or DC), and the students were asked to tick the item on the questionnaire which applied to them. The questionnaires were completed at the start of the programmes (Time 1), after eight months of programme delivery (Time 2) and six months after that (Time 3).

With the support of the Principal in each school, the study team worked closely with the key personnel involved in an SPHE delivery. A ‘support person’, usually the school guidance counsellor, was identified in each school, and teachers delivering both the SP and the EP gave information at the start of each lesson about how to access the support person. School personnel were given additional training in how to access further supports for young people at particular risk.

Statistical analyses

The first phase of analysis explored the comparability and representativeness of the study groups. This involved exploring any differences between the SP and EP groups at Time 1 and also comparing the sample retained at Time 3 with those who were lost to attrition. Chi-squared tests were used to explore gender differences, while a series of two-way ANOVAs were used to examine the interaction of condition (SP vs EP) and level of retention (Retained vs Lost) for age and Time 1 data.

For the main analyses, a series of complex factorial ANOVAS were conducted which included the variables of time (Time 1, Time 2 and Time 3), condition (SP and EP), gender (Male and Female) and caseness (Clinical/Borderline Range and Normal Range). The dependent or response variables were scored on the outcome measures including the SDQ, the CCSC and the non-standardized measures. These analyses were conducted in the context of the large sample size. In conducting the analyses, it was noted that for each of the variables (with the exception of the school ethos scale) a minimum individual cell size of 30 participants was maintained.

Results

Initial analyses

Data from all the three time points were available for 782 of the 1072 students who completed Time 1 measures, giving a retention rate of 72.9%. Data were excluded from those participants who had missed a data collection point. However, where individual items were missing, these data were replaced with the valid mean score for the participant on the remaining items. Chi-squared tests highlighted no significant relationship between gender and group membership (the EP and SP groups, χ2 = 0.047, N = 1072, p > 0.05) or between gender and retention in the study (those who were retained at Time 3 and those who were lost to attrition, χ2 = 0.02, N = 1072, p > 0.05). However, a significant relationship between group membership and retention (χ2 = 10.387, N = 1072, p < 0.05) and standardized residual analysis suggested that significantly more participants from the EP group were lost to attrition than expected. A series of two-way ANOVAS explored patterns of difference between the groups at Time 1 (see Table 1 for significant differences identified and the direction of the difference).

Table 1.

Comparison of samples at Time 1.

Variable	EP versus SP	Retain versus Lost	Interaction
Age	NS	F (1, 1064) = 6.278, p < 0.05* (R > L)	NS
Active coping	NS	F (1, 1068) = 12.767, p < 0.05* (R > L)	NS
Distraction	NS	NS	NS
Avoidance	NS	NS	NS
Support seeking	NS	NS	NS
School ethos	NS	NS	F (1, 981) = 5.274, p < 0.05*
Help-seeking	NS	NS	NS
Prosocial behaviour	NS	F (1, 1046) = 6.203, p < 0.05* (R > L)	NS
Hyperactivity	NS	NS	NS
Emotional difficulties	NS	NS	NS
Conduct problems	NS	F (1, 1046) = 4.907, p < 0.05* (R < L)	NS
Peer problems	F (1, 1044) = 6.202, p < 0.05* (EP > SP)	NS	F (1, 1044) = 3.992, p < 0.05*
Total difficulties	NS	F (1, 1044) = 4.732, p < 0.05* (R < L)	NS

Significant at 0.05.

Those participants who were retained in the study were somewhat older (average age = 13.7 years) than those who were lost to attrition (average age = 13.6 years). Given the mean scores, this has no practical significance. In examining the scores on the scales, no systematic patterns of difference were found.

However, as outlined in Table 1, differences were found between those who were retained in the study and those who were lost to attrition in level of active coping (those who were retained scored higher than those who were lost), prosocial behaviour (those who were retained scored higher than those who were lost), conduct problems (those who were retained scored lower than those who were lost) and total difficulties (those who were retained scored lower than those who were lost), and these patterns might suggest that those who were retained by the study showed a more positive profile. In addition, there was only one significant finding when the EP and SP groups were compared, with the participants in the EP condition scoring higher on peer problems than those in the SP. In addition, two significant interactions were identified (school ethos and peer problems), and these were further examined using tests of simple effects. Examining school ethos, the only significant contrast was between those who were retained and those who were lost to attrition for the EP condition (F = 5.977, df = 1, 981, p < 0.05), with the mean scores suggesting that those who were retained by the study reported higher levels of school ethos at Time 1. This analysis was repeated for the peer problems subscale where differences were found between those who were retained and those who were lost to attrition for the EP condition (F = 6.583, df = 1, 1044, p < 0.05) and between the EP and SP conditions for those who were lost to attrition. Examination of the mean scores suggested that in the EP condition, those who were lost to attrition had higher scores on the peer problems subscale than in the SP group. Although this might suggest that there was differential attrition from the EP group, it is important to note that the mean scores for all four subgroups (EP lost and retained, SP lost and retained) were well within the normal range for this subscale.

Main analyses

The main analyses were conducted on the ‘retained’ sample of 782 students. Of these, 52.9% of participants were male (n = 414) and 47.1% were female (n = 368). The group ranged in age between 12 and 16 years, with a mean age of 13.6 years (SD = 0.549). In the final sample, there were 361 students in the EP condition (46.2%) and 421 students in the SP condition (53.8%). Evidence of ‘caseness’ was explored within the group by examining the number of participants who scored in the borderline or clinical cut-off range of the total difficulties scale of the SDQ (i.e. > 15) at Time 1. Of the overall sample, 22.8% (n = 178) scored in the borderline or clinical range, with the remaining 588 participants falling in the normal range (75.2%; 2% of the sample had missing data on this scale). Looking at the breakdown of these groups across gender and condition, it was found that 92 of 354 students in the EP condition (25.9% of EP group, 46.7%/n = 43 male, 53.2%/n = 49 female) scored within this range, as did 86 of 412 students in the SP condition (20.9% of SP group, 34.9%/n = 30 male, 65.1%/n = 56 female). Chi-squared tests highlighted a significant relationship between gender and caseness (X = 12.514, N = 766, p < 0.05), with the standardized residuals indicating that there were fewer males (n = 73, 41%) and more females (n = 105, 59%) than expected in the overall caseness group. However, there was no evidence of a significant relationship between condition and caseness (X = 2.793, N = 766, p > 0.05).

Descriptive statistics for the sample that completed questionnaires at all three time points are presented in Table 2 for each of the scales.

Table 2.

Means and standard deviation for SP and EP groups at each time point.

	Time 1 Mean (SD)		Time 2 Mean (SD)		Time 3 Mean (SD)
Scale	SP	EP	SP	EP	SP	EP
School ethos	7.63 (2.42)	7.75 (2.26)	6.95 (2.84)	7.23 (2.81)	7.59 (2.52)	7.23 (2.75)
Help-seeking	31.41 (8.36)	30.62 (8.39)	30.52 (8.33)	30.02 (8.10)	31.92 (14.73)	27.92 (11.46)
Active coping	2.51 (0.46)	2.49 (0.44)	2.51 (0.48)	2.48 (0.45)	2.53 (0.50)	2.50 (0.46)
Distraction	2.32 (0.55)	2.25 (0.55)	2.22 (0.53)	2.17 (0.52)	2.19 (0.53)	2.22 (0.56)
Avoidance	2.34 (0.49)	2.32 (0.53)	2.32 (0.51)	2.31 (0.49)	2.29 (0.52)	2.26 (0.49)
Support seeking	2.13 (0.69)	2.21 (0.63)	2.11 (0.71)	2.15 (0.68)	2.18 (0.70)	2.17 (0.67)
Prosocial behaviour	6.94 (2.01)	7.29 (1.82)	6.88 (1.96)	7.03 (2.09)	6.95 (2.03)	7.19 (1.95)
Emotional difficulties	3.04 (2.43)	3.11 (2.15)	3.01 (2.35)	2.88 (2.22)	3.08 (2.44)	3.00 (2.27)
Conduct problems	2.65 (1.86)	2.93 (2.06)	2.53 (1.84)	2.71 (1.80)	2.49 (1.87)	2.59 (1.78)
Peer problems	1.37 (1.57)	1.39 (1.51)	1.37 (1.48)	1.40 (1.57)	1.36 (1.51)	1.35 (1.53)
Hyperactivity	4.29 (2.53)	4.56 (2.49)	4.30 (2.49)	4.76 (2.47)	4.22 (2.55)	4.73 (2.41)
Total difficulties	11.35 (5.61)	12.01 (5.65)	11.20 (5.49)	11.76 (5.34)	11.31 (5.60)	11.68 (5.44)

Looking at the results for the complex factorial ANOVAS, which included the variables of Time (Time 1, Time 2 and Time 3), condition (SP and EP), gender (male and female) and caseness (Clinical/Borderline Range and Normal Range) for each dependent variable, the main effects are presented in Table 3. At this level, significant changes over time were noted on school ethos, on two subscales of the CCSC (distraction and avoidance) and on four SDQ scales (excluding hyperactivity and prosocial behaviour). The patterns of difference suggest that scores generally decreased from T1 to T2 but then had either increased again by T3 or remained stable at T3. Interestingly, the only main effects for condition were on the help-seeking scale and SDQ peer problems subscale, with the SP group scoring higher than the EP group overall on both measures. Significant main effects for gender were also noted for the whole sample on a number of scales in Table 3. The dominant pattern was that females scored higher than males, with the exception of conduct problems. Finally, significant main effects for caseness were found on all scales except CCSC distraction. The dominant pattern for negative scales (e.g. SDQ and problematic coping) was that the students in the clinical/borderline range scored higher. However, the opposite was true for help-seeking and CCSC active and support seeking.

Table 3.

Table of significant main effects.

Variable	Significant main effects	Direction
School ethos	Time: F = 5.107, df = 2, 404, p < 0.05	Time: T1 > T2,
	Gender: F = 8.5, df = 1, 404, p < 0.05	Gender: F > M
	Caseness: F = 27.491, df = 1, 404, p < 0.05	Caseness: C > NC
Help-seeking	Condition: F = 4.362, df = 1, 729, p < 0.05	Condition: SP > EP
	Gender: F = 14.47, df = 1, 729, p < 0.05	Gender: F > M
	Caseness: F = 4.584, df = 1, 729, p < 0.05	Caseness: C < NC
Active coping	Caseness: F = 39.718, df = 1, 757, p < 0.05	Caseness: C < NC
Distraction	Time: F = 5.54, df = 2, 756, p < 0.05	Time: T1 > T2, Tl < T3
Avoidance	Time: F = 7.09, df = 2, 757, p < 0.05	Time: T1 > T3, T2 > T3
	Gender: F = 17.902, df = 1, 757, p < 0.05	Gender: F > M
	Caseness: F = 50.104, df = 1, 757, p < 0.05	Caseness: C > NC
Support seeking	Gender: F = 80.267, df = 1, 757, p < 0.05	Gender: F > M
	Caseness: F = 6.887, df = 1, 757, p < 0.05	Caseness: C < NC
Prosocial behaviour	Gender: F = 6.366, df = 1, 751, p < 0.05	Gender: F > M
	Caseness: F = 32.777, df = 1, 751, p < 0.05	Caseness: C < NC
Emotional difficulties	Time: F = 15.429, df = 2, 752, p < 0.05	Time: Tl > T2, Tl > T3
	Gender: F = 53.561, df = 1, 752, p < 0.05	Gender: F > M
	Caseness: F = 225.519, df = 1, 752, p < 0.05	Caseness: C > NC
Conduct problems	Time: F = 34.924, df = 2, 751, p < 0.05	Time: Tl > T2, Tl > T3
	Gender: F = 54.008, df = 1, 751, p < 0.05	Gender: F < M
	Caseness: F = 297.351, df = 1, 751, p < 0.05	Caseness: C > NC
Peer problems	Time: F = 4.654, df = 2, 750, p < 0.05	Time: T1 > T2, T1 > T3
	Condition: F = 6.662, df = 1, 750, p < 0.05	Condition: SP > EP
	Gender: F = 14.199, df = 1, 750, p < 0.05	Gender: F > M
	Caseness: F = 94.845, df = 1, 750, p < 0.05	Caseness: C > NC
Hyperactivity	Gender: F = 231.713, df = 1, 752, p < 0.05	Gender: F > M
	Caseness: F = 32.777, df = 1, 752, p < 0.05	Caseness: C > NC
Total difficulties	Time: F = 30.168, df = 2, 750, p < 0.05	Time: T1 > T2, T1 > T3
	Caseness: F = 599.874, df = 1, 750, p < 0.05	Caseness: C > NC

Building on the analysis of main effects, a key aspect of the analysis was the identification of the impact of condition, with the intention to identify any differences between the SP and EP groups. Given the multi-factorial nature of the analyses conducted, the authors were able to identify and explore any interaction involving the condition variable. Table 4 shows the findings for significant interactions with the condition variable. The two-way interactions were further analysed using tests of simple effects, highlighting that at T3 the EP group showed lower levels of help-seeking behaviour than the SP group and that for the SP group distraction scores had dropped significantly at T2 and remained stable at T3. Both of these findings might suggest that the SP group showed greater change.

Table 4.

Table of significant interactions with condition.

Variable	Significant interaction(s) with condition	Direction
Help-seeking	Time × cond.	At T3 only
	F = 4.29, df = 2, 729, p < 0.05	EP < SP
Distraction	Time × cond.	For SP only
	F = 4.204, df = 2, 756, p < 0.05	T1 > T2, T1 > T3
Support seeking	Time × cond. × case F = 5.501, df = 2, 757, p < 0.05
Prosocial behaviour	Time × cond. × case	For caseness group
	F = 3.553, df = 2, 751, p < 0.05	Sig. Int. time × Cond.
Peer problems	Cond. × case	For EP and SP
	F = 14.331, df = 2, 750, p < 0.05	C > NS
Hyperactivity	Time × cond. × gender × case
	F = 3.111, df = 2, 752, p < 0.05
Total difficulties	Time × cond. × gender × case
	F = 4.789, df = 2, 750, p < 0.05

Significant three-way interactions were found between the variables of time, condition and caseness for support seeking and prosocial behaviour. Given the limitations of the analysis programme used with regard to unpicking a three-way interaction, the analysis was re-run separating out the caseness and no caseness groups (reducing alpha to 0.025), and it was found that for both scales, there was a significant interaction between time and condition for those with a clinical level difficulty at T1. However, test of simple effects identified no significant differences for prosocial behaviour, and a single significant difference between EP and SP at T1 with those in the SP showed lower levels of support seeking.

Finally, significant four-way interactions were found between time, condition, gender and caseness for the SDQ total difficulties scale and the SDQ hyperactivity subscale. While it is difficult to unpick the specific nature of these interactions, when this analysis was run separately for the caseness and no caseness group a number of notable patterns were found. Firstly, looking at the SDQ total difficulties scale, it was found that the interaction between time, condition and gender was only significant for the caseness group. Figure 1 provides some insight into this interaction. The graph suggests that males in the caseness group who are in the EP condition show better change over time than those who are in the SP condition and when compared to females in both groups.

Figure 1.

Plot of mean scores on total difficulties for participants in the caseness group.

Exploring the interaction between time, condition, gender and caseness for hyperactivity in the same way a similar pattern emerged. Although there was no significant interaction between time, condition and gender for those in the no caseness group, there was a trend (significance p = 0.052) indicating a three-way interaction between time, condition and gender for those in the caseness group. Looking at Figure 2 while males in the SP condition seem to be lower than others at the start, males in the EP condition show better change over time than females in both groups (EP and SP), and by T3 they are similar to males in the SP condition who have remained fairly stable over time.

Figure 2.

Plot of mean scores on hyperactivity for participants in the caseness group.

Discussion

This study showed few differences between the groups over time. The only significant differences between the groups as a whole were in terms of help-seeking, with students in the SP showing greater improvement over time than those in the EP, while those in the enhanced group showed greater reduction in peer problems. ‘At risk’ boys in the EP showed significantly greater reductions in their emotional and behavioural difficulties, their hyperactivity and their total difficulties than those in the SP. Thus, the main hypotheses of the study were not supported, but the EP appeared to selectively benefit ‘at risk’ boys in terms of reducing their behavioural difficulties, but without showing improvements in their coping strategies.

It could be argued that the interventions provided to the two groups were very similar and that the ‘dose’ provided to the EP group was too small to have any significant overall effect on outcome.

The strength of the study lies in the fact that it was an RCT, carried out in the ‘real world’, with a limited budget for a study of this size. It also has a number of limitations. The lack of a ‘no intervention’ group meant that there is no way of knowing whether the changes found were not simply due to the passage of time, although this is unlikely as a study by Hawes and Dadds (2004) of Australian children reported stability of SDQ scores over a 12-month period, while Goodman (2001) showed a test–retest reliability over a six-month period of 0.71 for the total difficulties score of the SDQ. Any improvements over time could also be related simply to the schools’ participation in a ‘research’ study (Hawthorne effect), rather than any specific effect of the programmes, although this would be expected to produce similar changes in both groups. A further limitation may be the fact that teachers in the EP condition had more input from the research team than those in the SP condition. They had training in delivery of Working Things Out through SPHE, and were required to complete fidelity checks indicating that they had delivered the programme in keeping with the manual.

The study was confined to those for whom measures were available for all the three time points, with results suggesting that they may be better functioning than those who were not retained in the study. It has been shown in mental health research that those who drop out of follow-up studies, or who cannot be traced, have higher levels of psychopathology than those who remain (Cleary, Fitzgerald, & Nixon, 2004), and this study, which showed that those who were lost to attrition had higher levels of behaviour problems, lower prosocial behaviour and lower active coping strategies than those who were retained, supports that finding. Significantly more students were lost from the EP condition than from the SP condition, which may be related to their higher levels of peer problems at the outset, but may be due to unidentified reasons, including the possibility that participation in the programme contributed in some way to their loss.

Limited resources meant that there was an absence of ‘third party’ information about the students. While students themselves are probably the best people to report on their help-seeking behaviour, peer relationships, emotional difficulties, coping styles and their view of school ethos, it would have added strength to the study to have a report from a parent or a teacher on their behavioural difficulties and prosocial behaviour.

Despite these limitations, it was encouraging that boys scoring in the ‘at risk’ range on the total difficulties score and hyperactivity scale of the SDQ showed improvement over time and that improvement was significantly greater in the ‘enhanced’ group. While it is well known that high scorers tend to regress towards the mean, this is unlikely to be the full explanation, given the differences between the groups. ‘At risk’ young male adolescents are at high risk of poor outcomes in terms of anti-social behaviours, underachievement at school and work, unstable relationships, mental health problems and poor parenting in adult life (Bradley, Henderson, & Monfore, 2004; Hill, 2003; Moffitt, Caspi, Harrington, & Milne, 2002). Although some evidence-based interventions exist for high-risk youth, there are difficulties in delivering these in the ‘real world’ of face-to-face work with ‘at risk’ young men, who are notoriously difficult to engage in services (Whittaker, 2009). Thus, any intervention which appears to benefit them is worthy of consideration, particularly one which keeps them in the mainstream, as aggregating high-risk youth into groups may be associated with an increase in their problem behaviours (Dishion & Andrews, 1995; Poulin, Dishion, & Burraston, 2001), although this has been disputed (Weiss et al, 2005). It is possible that the approach of the ‘Working Things Out’ stories, with their eye-catching graphics and ‘real’ voices, impacted on these ‘at risk’ young people in a way in which the SP did not.

These results were both encouraging and disappointing. It was encouraging that students’ emotional/behavioural difficulties lessened over time, whether this was due to a specific intervention or not. It was disappointing that ‘help-seeking’ improved less in the ‘enhanced’ group, where encouragement and modelling of ‘help-seeking’ were an important theme. The selective improvement seen in ‘at risk’ boys in the EP is exciting, but needs further study to see if this finding is replicated.