SPARX-R computerized therapy among adolescents in youth offenders' program: Step-wise cohort study.

BACKGROUND: Youth offenders have high rates of unmet mental health needs, including elevated rates of subclinical or clinical depression. Computerized cognitive behavioral therapy (cCBT) has been shown to be effective for depression, and cognitive behavioral therapy (CBT) is among the most effective psychological treatments for offence related behaviours. We planned to evaluate the impact of SPARX-R 1.0 (the first iteration of a revised version of SPARX cCBT) for adolescents in a community day program (Mentoring Youth New Directions or MYND) for male recidivist youth aged between 14 and 17 years. Recruitment and retention in the trial were lower than anticipated. In this brief report we present main findings and discuss implications.
METHODS: We developed a stepwise cohort design to investigate the acceptability and effectiveness of SPARX-R in a complex, real-world setting. Participants were allocated to the MYND program only (treatment as usual), or MYND with the addition of SPARX-R. All adolescents referred to MYND within a specified period were assigned to one of four social workers, as per usual practice. Each social worker was randomized to begin SPARX-R with consenting new clients from one of four time points. Assessments were completed within the first two weeks of commencing the MYND program and then at 10 and 20 weeks after commencement. We solicited brief feedback on SPARX-R from young people and staff who used it.
RESULTS: Of 64 eligible youth who began MYND during the trial period, 51 consented but 25 stopped attending MYND despite court orders or because their court orders were changed. Nineteen participants were randomized to SPARX-R but only two completed two or more levels of the 7-level program, so it was not possible to evaluate the impact as planned. The four participants who provided feedback were indifferent or negative about SPARX-R. Staff advised that technical difficulties (such as loading or saving problems) were off putting and that SPARX-R was slow and not appealing to their clients.
CONCLUSIONS: Computerized CBT was not successfully implemented in this group, highlighting challenges in retention in this non-residential justice program. The findings also indicate that computerized therapies of proven acceptability and effectiveness in one setting may be unappealing in another. Implementation and equity efforts need to consider and test the specialist needs of diverse groups.

Introduction

Youth offenders have high rates of unmet mental health needs, including elevated rates of subclinical or clinical depression and emotional distress or dysregulation (Chitsabesan et al., 2006; Teplin et al., 2002; Townsend et al., 2010). Untreated, these can lead to ongoing mental distress, increased risk of harming oneself or others, and increased rates of re-offending (Birmaher et al., 1996; Mallett and Boitel, 2016; Rao and Chen, 2009). Cognitive behavioral therapy (CBT) appears to be among the most effective psychological therapies for adolescent depression (Thapar et al., 2012), and for offence related factors (Armelius and Andreassen, 2007; Kip et al., 2018). However, implementing face-to-face CBT in programs for youth offenders is challenging due to a shortage of therapists with the requisite skills. Computerized CBT (cCBT) has been shown to be effective and acceptable in adolescent populations (Ebert et al., 2015; Christensen et al., 2011; Merry et al., 2012), and could increase access to therapy among hard-to-reach groups (Fleming et al., 2019a). However, few cCBT programs for adolescents have been trialled outside of school or health care settings, and we found just one published trial of cCBT among youth offenders (Wannachaiyakul et al., 2017). In this trial, usual activities in combination with a purpose built cCBT intervention were found more effective than usual activities alone in reducing symptoms of depression among depressed youths in a residential juvenile vocational training center in Thailand.

Members of our team developed a cCBT program (SPARX) and tested this in a randomized controlled trial of 187 adolescents seeking help for depressive symptoms (Merry et al., 2012). SPARX was as effective as treatment as usual (which primarily comprised face-to-face counselling) on measures of depression and psychological functioning. Team members also tested SPARX in a pragmatic trial with adolescents in Alternative Education programs, in which students have been excluded or alienated from mainstream schools and are considered to have major behavioral difficulties in classroom settings. Among participants with depressive symptoms at baseline, those who received the SPARX program had significant reductions in symptoms compared with those in the waitlist control group (Fleming et al., 2012a). In follow-up interviews, participants, including those without depressive symptoms, indicated that they had found SPARX useful. Commonly cited benefits included feeling calmer and less angry, and reduced involvement in fights or disciplinary procedures. Students felt that targeting cCBT to those with symptoms was problematic because of potential stigma, and that cCBT should be made available to all young people in programs such as their own whether or not they were feeling down, as this could help them develop skills to deal with potential future problems as well as current challenges (Fleming et al., 2016). In pre-intervention interviews, youth workers and justice program providers expressed enthusiasm about the potential of cCBT (Fleming and Merry, 2013). They stated that their clients were underserved by mental health services, in part because clients did not wish to speak with unfamiliar professionals and found existing service models unappealing (Fleming and Merry, 2013). Despite this lack of engagement, providers reported that many of their young clients expressed the desire to make positive change, and providers were eager to support this effort (Fleming and Merry, 2013).

Despite known difficulties in conducting research in justice settings (Abrams, 2010; Pitts and Smith, 2007), it is important to examine whether cCBT is acceptable and satisfactory to youth in justice programs, and whether implementation in these settings is feasible and effective. There is growing recognition of the importance of evaluating the acceptability and impact of interventions with diverse populations under the conditions in which they are intended to be used (Munro and Bloor, 2010; Palinkas et al., 2011). Such research questions require thoughtful attention to scientific methods to maximise both validity and rigor (Oakley et al., 2006; Thorpe et al., 2009). The present study was designed to test the feasibility and acceptability of a modified version of SPARX (SPARX-R) among youth referred to a compulsory day based rehabilitation program for offending (Mentoring Youth New Directions; MYND), and to assess the effectiveness of SPARX-R in this complex, real-world setting. A secondary aim was to evaluate the effectiveness of the MYND program.

Methods

Participants

Male adolescents (aged 14–17 years) referred by the court to the MYND program between 1st August 2013 and 6th November 2014 were assessed for inclusion in the study. Adolescents were eligible for inclusion if they had been referred to MYND for a minimum of 10 weeks, were able to provide written consent (and, if under 16 years, had not been excluded by a parent or guardian), and had sufficient English language to understand the SPARX-R program. Parental consent was not required, but parents or guardians of those under 16 were informed of the study and could opt to have their young person excluded. Exclusion criteria were minimal, but youth were not enrolled if they: were not attending the program, had intellectual or physical limitations that would prevent them from using the computer program, or had high mental health needs that could compromise their safety.

Design

The study design was developed with MYND managers and staff to fit client needs and their workplace procedures and priorities. Adolescents referred to MYND were allocated to one of four social workers as part of MYND standard practice. (That is, MYND had four social workers who were the primary service providers to clients, working one to one and/or with family members and occasionally in small groups with clients). Participants in the study received either the MYND program only (treatment as usual, hereafter ‘MYND’), or MYND and SPARX-R together (hereafter ‘SPARX-R’). Using a stepwise design, each social worker was randomly allocated to begin SPARX-R with all their consenting new clients from one of four time points spread evenly through the study period. This design was selected to minimize contamination, as staff had indicated in preliminary consultation that they would like to learn CBT skills from SPARX to implement with all their clients in their everyday practice. Social workers were trained to carry out the assessments with their clients as they advised that participants might find it challenging to trust the researchers, who were relative strangers to them.

Interventions

The MYND program is a compulsory community rehabilitation program for males aged 14–17 years who have been referred by statutory authorities for offending; it is offered in one site in Auckland New Zealand. There were a total of four social workers working at MYND for the study period, all of whom participated in the study. MYND is based on one-to-one mentoring, goal-setting, problem-solving, and task-centred social work and youth development activities completed with a specialist social worker in the participant's home or community or the MYND premises. Each participant works with their social worker to determine specific activities and goals. Adolescents must participate for between 6 and 30 h per week for a designated period, typically of at least 10 weeks. MYND is regarded as an intensive community based program, and an alternative for young people who would otherwise be given custodial sentences, frequently for convictions related to violence.

SPARX-R is a version of the evidence-based SPARX computerized cCBT program which has been described previously (Merry et al., 2012). In brief, SPARX uses a virtual therapist (‘the guide’) and play based exploratory learning in a game like environment, to teach and rehearse skills for relaxation; activity scheduling; problem solving; emotion regulation; identifying and replacing negative cognitions and improving social skills (Merry et al., 2012). SPARX-R involves minor modifications to SPARX, to increase its appeal and relevance to young people to who do not identify as feeling depressed and to reduce content specific to those in high school (Fleming et al., 2019b). For example, SPARX begins with a “Guide” character (a virtual therapist), who states that the purpose of the program is to help young people “who feel down or depressed”. While SPARX-R, the Guide states: “This version of SPARX was made to help young people who are having hassles and feeling down, stressed or angry a lot of the time. Even if you are doing fine, SPARX can help strengthen your skills for dealing with problems when they do come along.” Participants allocated to SPARX-R were invited to complete the computer-administered, self-directed SPARX-R intervention (cCBT) alongside their social worker during their standard MYND program hours. No additional program hours were provided; hence SPARX-R could displace other MYND activities. SPARX-R comprises seven modules completed sequentially. Participants were asked to complete no more than two modules per week and at least one module per fortnight, using a laptop in an appropriate setting (e.g., MYND office, home, or other private space).

Procedure

Assessments were made at baseline (2 weeks after commencing MYND and prior to beginning SPARX-R) and at 10 and 20 weeks after commencing MYND. Assessments were not taken earlier because it was felt that young people might under-disclose problems until they developed some trust in their social worker (e.g. see Pitts and Smith, 2007). Primary outcomes were: change in depressive symptoms, measured with the Reynold's Adolescent Depression Scale, Second Edition (RADS-2) (Reynolds, 2002); and change in anti-social cognitions, measured with the How I Think (HIT) questionnaire (Barriga et al., 2001). Secondary outcomes were: change in depressive symptoms, measured with the clinician-rated Children's Depression Rating Scale Revised (CDRS-R) (Poznanski and Mokros, 1996); change in anger, measured with the Beck Anger Scale (Beck et al., 2005); and change in responses to help-seeking and hurting others questions. Help-seeking questions asked whether participants would seek help from an adult if they were depressed, suicidal or felt they may hurt someone. Hurting others questions asked how often in the past 10 weeks participants had physically hurt someone on purpose, been in a serious fight or carried a weapon. Participants who had used SPARX-R were invited to give feedback about their experience and how useful they had found it via a simple pen and paper questionnaire (items: I liked SPARX-R; SPARX-R was helpful for me; SPARX-R could be helpful for other young people in programs like MYND; Would doing SPARX-R make young people more likely to ask for help if they had problems? Should we keep using SPARX-R in MYND? How could we improve SPARX-R? Response options for the closed questions were on a five-point Likert scale, from ‘Not at all’ to ‘Yes a lot’).

Social workers were invited to give feedback at a focus group carried out by one of the researchers (BG) at the MYND premises. The focus group followed a semi-structured interview guide, which invited discussion on the study procedure (How did the assessment interviews go? What was it like doing these? Did you feel the young people were able answer honestly? What was it like inviting the young people to participate in the study? Were there barriers to retaining them in the study?), using SPARX-R in this setting, and potential improvements (How was it using SPARX-R on the computer with the participants? What were the barriers? What might help improve SPARX-R? Other suggestions?). The focus group was recorded, field notes were kept and following the focus group BG and TF summarised results under each of the research questions. Summarised results were confirmed with the social workers.

A sample size of 100 youth participants was sought. Eighty participants would enable a moderate effect size (a change in raw score on the RADS-2 of at least 5.5 points) to be shown as statistically significant (2-tailed α = 0.05) with 80% power. MYND providers advised that most adolescents who attend the MYND program do so for between 20 and 26 weeks. Hence, a sample size of 100 was sought, to allow for 20% drop out.

Ethics

The trial was prospectively registered with the Australia and New Zealand Clinical Trials Registry, and ethical approval was gained from the New Zealand Health and Disability Ethics Committee, a ministerial committee established under the New Zealand Ministry of Health (Northern Y committee, reference: 13/NTA/79.). Parents or guardians were informed of the study and could ask for their under 16 year old to be excluded. Adolescents provided their own written consent.

Results

As shown in Fig. 1, of the 104 young offenders (all male) referred to MYND during the study period, 40 were not eligible to participate, most often because they did not commence or stopped attending MYND prior to baseline assessments, despite court orders. Of those invited, 51 consented but ten of these did not begin the study, again most often because they stopped attending the program. Forty completed a baseline assessment and were allocated to receive MYND or SPARX-R.

Fig. 1

Consort diagram.

*Young person may be referred to MYND by the courts more than once in the time period. For example a young person may complete their orders and be referred again due to additional offending.

Participant characteristics and baseline scores on standardised measures are shown in Table 1. All were aged 14–17 and all but one were of indigenous Māori or Pacific Island ethnicity. Seven participants (five in the MYND only and two in the SPARX-R group) had scores indicating possible depressive symptoms on CDRS-R (T score above 30: Poznanski and Mokros, 1996). No participants had scores indicating possible symptoms of depression on RADS-2 alone (RADS-2 score over 77; Reynolds, 2002). Thirteen participants (six in MYND only and seven in the SPARX-R group) scored 19 or less on the CDRS-R. Poznanski and Mokros (1996) indicate that scores at this level are uncommonly low and suggest that these may be associated with pervasive denial.

Table 1

Baseline descriptive statistics for all participants (n = 40).

	MYND group (n = 21)	SPARX group (n = 19)
Age (years)
14	n = 2	n = 2
15	n = 8	n = 4
16	n = 8	n = 9
17	n = 3	n = 4
Ethnicity
Māori	66.6% (n = 14)	68.4% (n = 13)
Pacific	28.6% (n = 6)	31.6% (n = 6)
NZE	4.8% (n = 1)	0
RADS-2	50.43(10.92)	51.63(12.08)
CDRS-R	24.15(5.50)	22.06(7.21)
HIT	3.45(0.88)	3.29(0.74)
Beck Anger Scale	12.52(6.65)	11.68(8.36)

Note: Unless expressed as a percentage, values are means and standard deviations.

MYND = Mentoring Youth New Directions; NZE = New Zealand European; RADS-2 = Reynold's Adolescent Depression Scale - 2; CDRS-R = Child Depression Rating Scale - Revised; HIT = How I Think Questionnaire.

Of the 40 participants who completed a baseline assessment, 14 were no longer attending MYND by the 10 week assessment as they had changes in plans ordered by the court or were missing and could not be located by social workers. As shown in Fig. 1, a further seven participants were still considered enrolled in MYND at the time of assessment but could not be located to complete this assessment. In sum, only ten participants allocated to MYND alone and nine allocated to SPARX-R completed the ten-week assessment. Hence, it was not possible to conduct between-group comparative analyses to test the effectiveness of MYND with SPARX-R compared with that of MYND alone nor to evaluate the effectiveness of MYND alone as planned.

Only two of the 21 participants allocated to SPARX-R completed more than two of the seven levels. Four participants assigned to SPARX-R completed the satisfaction questionnaire. Two reported that they did not like the SPARX-R program, and the other two reported that it was okay. One participant reported that SPARX-R “was not my thing” and “felt babyish”, and another participant said that there was “too much talking”.

All four social workers who delivered the program attended the focus group. They described carrying out the study as challenging. Social workers reported that participants would say that they were coming but would not arrive or could not be found when the social worker arrived to pick them up. They judged that this reflected complexities of working with young people in a non-residential justice program and that this occurred regardless of the study. They felt that it took time to build rapport and that, even with the two-week delay between beginning the MYND program and completing the baseline assessments, young people were not always open about their personal wellbeing, and may have underreported their problems at baseline. One social worker advised that a participant had completed four modules. Aside from this individual, the social workers reported that most participants allocated to SPARX-R only did the first session. They advised that operational and technical challenges were off putting: getting a private space and time on the laptop was tricky, and some had tried doing this in cars with the client. The computer program was slow to load and some had difficulties with saving progress. Aside from technical difficulties, social workers agreed that the guide (virtual therapist), talked too much and that “these boys are used to operating at a faster level”. They advised that many of the participants played commercial video games and that SPARX-R was of lower technical quality (e.g. was harder to move around and had poorer graphics) and was much slower to play than these games. Some considered that it might be more appropriate for younger adolescents and were interested in trying it with participants' younger siblings.

On the basis of the limited engagement in SPARX and the largely neutral or negative feedback from youth and social workers, we judged that SPARX-R 1.0 was not shown to be acceptable or feasible in this setting.

Discussion

Computerized CBT was not successfully implemented in this trial with male adolescents in a community based program (MYND) for young offenders. Fewer adolescents began MYND during the study period than anticipated. Of those who began, many stopped attending despite court orders or because they had changes in court orders (due to discharge by the courts or re-arrest). Furthermore, those allocated to SPARX-R did not engage with the computerized therapy. Although we had carried out consultation with MYND providers while planning the trial, and had found positive results with young people in similar communities who had been excluded from mainstream education, it was not possible to complete the study as planned. These findings highlight the importance of conducting feasibility studies prior to conducting trials. Secondly, they highlight that cCBT programs found acceptable and effective in one group or setting may have quite different results in another, even related, setting.

Feasibility studies are carried out prior to a larger trial to identify whether the trial study is possible and assess critical characteristics, such as participant willingness and potential participant numbers (Arain et al., 2010). A feasibility study was not planned in this case. SPARX had positive effects and good acceptability among staff and students in Alternative Education Centres (Fleming et al., 2012a, Fleming et al., 2012b, Fleming et al., 2016). All of the participants in Alternative Education trial were between 13 and 16 years old, and most were Māori or Pacific males who had been excluded from mainstream education, often due to their behaviour and violence. Many lived in the neighbourhood that the MYND program was based in. In particular, participants in that trial commented that SPARX stopped them from fighting or hurting others. One commented that:

“You should give it to people in prison, they'll stop killing each other”.

There had also been positive interest from youth workers and frontline workers, including in some youth justice settings (Fleming and Merry, 2013). Despite these positive indications and pre-trial consultation, the trial did not progress as planned. A feasibility study should have identified this. This said, community organisations and court referral processes do change and a feasibility study may not have identified all issues. For example, earlier MYND referral data suggested that higher retention in the program could be expected.

The finding that cCBT was unappealing in this setting and yet engaging in other settings is of interest. In a trial of young people seeking help from primary care, school health, or school counselling, adherence to SPARX was good and satisfaction was high (Merry et al., 2012). There was high uptake of and engagement with SPARX by youth attending alternative education, including among those without symptoms of depression (Fleming et al., 2016), and high interest was reported among Māori and sexual minority youth (Shepherd et al., 2018; Lucassen et al., 2015). The program used in this study was the first iteration of revised version, SPARX-R 1.0. This version was delivered using CDROMS or on memory sticks and there were a number of technical challenges including with saving progress. SPARX-R 1.0 was also tested with limited success in Alternative Education Centres in Ireland (Kuosmanen et al., 2017); here, too, technical challenges such as slow loading speed and problems with saving were reported as off putting. Programming challenges have since been addressed and an online version of SPARX-R was found to prevent depression among final year high school students in a cluster RCT involving 540 adolescents (Perry et al., 2017). Perhaps we tested too early and should have trialled the intervention once the usability and technical issues had been better addressed.

Our participants were ordered by the courts to participate in the MYND program. Although we aimed to be very clear that participation in the study and use of SPARX-R was optional, much of their experience at MYND was under compulsion and this may have affected their motivation and interest. The only other trial we identified of cCBT in a youth justice setting found that a purpose built cCBT program (few details regarding this program were described by the authors, but it was not described as using game features) was effective for depressive symptoms among incarcerated youth (Wannachaiyakul et al., 2017). A recent review highlighted that eHealth interventions are promising in forensic settings, in part as inmates may not wish to discuss intimate issues with staff (Kip et al., 2018). In contrast, our participants were not incarcerated. Further, the MYND program seeks to recruit staff who can become close to the participants, understand their cultures and life experience, and help them build strengths, often via one on one coaching, using humour, personal challenges or fitness, and via cultural or other relational connections. It is possible that a computerized therapy, potentially displacing other activities, and requiring staff to sit with a young person at a laptop and work through a computer program using cCBT skills, was a poor fit.

We received limited feedback about SPARX-R in this study, however there was indication that it was “too babyish”, too slow and had too much talking. Interestingly, it was successful among older adolescents in other trials. Possibly issues such as compulsion were at play, or the fit or style was not satisfactory for these young people beyond age considerations alone. Feedback did include that SPARX-R did not meet the usual standards of video games played by participants. It is possible that the fantasy game like graphics and presentation led to expectations that were not fulfilled.

Strengths and limitations

This paper has a number of limitations. In particular, recruitment and retention were much lower than anticipated and we were unable to complete planned effectiveness analyses. We had limited youth feedback and just one brief staff focus group. Further, as the study was carried out with young people in one specific program for youth offenders in one location, generalisability is limited. However, as our research group and others have reported positive or promising findings for SPARX and SPARX-R with various populations (Fleming et al., 2012a; Lucassen et al., 2015; Merry et al., 2012; Perry et al., 2017; Shepherd et al., 2018) we consider it important to present the current results, despite having limited information as to why SPARX-R was not appealing.

Conclusion

Our findings highlight challenges with conducting research in a day based program for youth offenders where attendance was irregular and drop out from the day based program was high. The findings underscore the importance of conducting feasibility trials when planning research in challenging settings. SPARX-R (version 1.0) cCBT was not engaging for these participants, highlighting that it cannot be assumed that cCBT proven acceptable and effective in some settings will be appealing in other settings or to users among whom it has not been tested. Implementation and equity efforts need to consider and test the specialist needs of diverse groups.

Declaration of competing interest

Theresa Fleming, Karolina Stasiak, Matthew Shepherd and Sally Merry are co-developers of SPARX and SPARX-R. The IP for SPARX and SPARX-R is owned by Uniservices at the University of Auckland and the co-developers can benefit from the commercialisation of it outside of New Zealand. Bridgette Gillham and Julie Moore are employed by The Graeme Dingle Foundation, MYND is fully funded by the New Zealand the government but it operates as part of the Graeme Dingle Foundation.

This research was supported by a research grant to the Department of Psychological Medicine, University of Auckland, from the Graeme Dingle Foundation (previously known as the Foundation for Youth Development).